Oct 31 03:27:40.265571 ip-10-0-130-19 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 03:27:40.655900 ip-10-0-130-19 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 03:27:40.655900 ip-10-0-130-19 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 03:27:40.655900 ip-10-0-130-19 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 03:27:40.655900 ip-10-0-130-19 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 03:27:40.662159 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662022    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:40.662159 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662161    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662166    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662169    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662173    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662176    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662179    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662182    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662184    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662187    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662191    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662193    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662196    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662198    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662201    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662203    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662206    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662209    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662211    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662214    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662216    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:40.662229 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662219    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662221    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662224    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662228    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662231    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662234    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662236    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662239    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662242    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662244    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662247    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662249    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662252    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662255    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662257    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662259    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662262    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662280    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662284    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662288    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:40.662731 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662292    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662296    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662298    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662301    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662304    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662306    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662309    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662311    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662314    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662317    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662319    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662321    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662324    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662327    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662330    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662333    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662336    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662339    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662342    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662345    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:40.663188 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662348    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662350    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662353    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662355    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662358    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662360    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662363    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662367    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662369    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662372    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662375    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662377    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662380    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662382    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662385    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662387    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662390    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662392    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662395    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:40.663673 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662397    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662400    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662403    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662405    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662407    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662410    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662413    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662416    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662419    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662421    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662424    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662427    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662430    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662432    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662434    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662437    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662441    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662445    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662450    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:40.664154 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662455    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662458    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662461    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662463    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662466    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662468    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.662471    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663283    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663290    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663292    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663296    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663299    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663302    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663306    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663309    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663311    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663314    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663316    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663319    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663322    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663325    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:40.664630 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663328    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663330    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663333    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663337    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663339    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663342    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663345    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663347    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663350    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663352    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663355    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663357    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663360    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663362    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663365    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663368    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663370    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663373    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663375    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:40.665567 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663378    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663380    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663383    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663385    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663388    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663391    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663393    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663396    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663399    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663401    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663404    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663407    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663409    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663412    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663415    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663417    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663419    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663422    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663425    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:40.666023 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663428    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663430    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663433    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663435    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663438    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663440    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663443    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663445    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663450    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663455    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663458    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663461    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663464    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663466    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663470    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663473    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663475    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663478    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663480    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:40.666521 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663483    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663486    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663488    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663490    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663494    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663497    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663500    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663504    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663507    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663510    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663513    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663515    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663518    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663521    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663524    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663528    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663530    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663532    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663535    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663537    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:40.666981 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663540    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663542    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663545    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663547    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663550    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663552    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663554    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663557    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663559    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663562    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663565    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663568    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663570    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663573    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.663575    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664076    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664081    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664084    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664087    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:40.667465 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664089    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664092    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664095    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664097    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664100    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664105    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664109    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664111    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664114    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664117    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664119    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664122    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664125    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664127    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664130    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664133    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664135    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664138    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664140    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:40.667902 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664143    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664145    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664148    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664151    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664153    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664157    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664161    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664164    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664166    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664169    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664172    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664175    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664178    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664180    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664183    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664185    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664188    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664190    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664193    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664202    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:40.668367 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664204    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664207    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664209    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664212    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664215    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664217    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664220    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664222    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664224    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664227    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664229    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664232    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664234    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664236    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664239    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664242    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664244    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664247    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664249    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:40.668838 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664252    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664255    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664257    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664260    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664277    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664280    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664282    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664285    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664288    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664290    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664293    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664295    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664298    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664301    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664304    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664306    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664309    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664311    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664314    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664316    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:40.669299 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664318    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664321    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664323    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664326    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664328    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664330    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664335    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664338    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664340    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664343    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664346    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664349    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664351    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664354    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664356    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664359    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664362    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664364    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664367    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664369    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:40.669810 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664372    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664374    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664376    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.664379    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665242    2580 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665251    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665260    2580 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665283    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665291    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665295    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665299    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665303    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665306    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665309    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665312    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665316    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665319    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665321    2580 flags.go:64] FLAG: --cgroup-root=""
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665326    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665329    2580 flags.go:64] FLAG: --client-ca-file=""
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665332    2580 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665335    2580 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665339    2580 flags.go:64] FLAG: --cluster-domain=""
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665342    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 03:27:40.670289 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665345    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665349    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665352    2580 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665356    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665359    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665362    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665365    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665368    2580 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665371    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665374    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665377    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665380    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665384    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665387    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665391    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665395    2580 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665398    2580 flags.go:64] FLAG: --enable-server="true"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665401    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665405    2580 flags.go:64] FLAG: --event-burst="100"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665408    2580 flags.go:64] FLAG: --event-qps="50"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665411    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665414    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665417    2580 flags.go:64] FLAG: --eviction-hard=""
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665421    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 03:27:40.670857 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665426    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665429    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665433    2580 flags.go:64] FLAG: --eviction-soft=""
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665436    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665439    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665441    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665444    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665447    2580 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665450    2580 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665453    2580 flags.go:64] FLAG: --feature-gates=""
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665457    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665460    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665463    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665467    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665469    2580 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665473    2580 flags.go:64] FLAG: --help="false"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665475    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665479    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665482    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665484    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665488    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665491    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665494    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 03:27:40.671488 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665497    2580 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665506    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665509    2580 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665512    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665515    2580 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665518    2580 flags.go:64] FLAG: --kube-reserved=""
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665520    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665523    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665526    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665530    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665533    2580 flags.go:64] FLAG: --lock-file=""
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665539    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665542    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665545    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665551    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665554    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665556    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665559    2580 flags.go:64] FLAG: --logging-format="text"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665562    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665565    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665568    2580 flags.go:64] FLAG: --manifest-url=""
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665571    2580 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665576    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665579    2580 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665583    2580 flags.go:64] FLAG: --max-pods="110"
Oct 31 03:27:40.672055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665586    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665589    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665592    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665594    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665597    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665600    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665602    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665609    2580 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665612    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665621    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665624    2580 flags.go:64] FLAG: --pod-cidr=""
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665627    2580 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665629    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665632    2580 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665635    2580 flags.go:64] FLAG: --port="10250"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665638    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665642    2580 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-02580f19870119d71"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665645    2580 flags.go:64] FLAG: --qos-reserved=""
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665649    2580 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665652    2580 flags.go:64] FLAG: --register-node="true"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665655    2580 flags.go:64] FLAG: --register-with-taints=""
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665659    2580 flags.go:64] FLAG: --registry-burst="10"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665661    2580 flags.go:64] FLAG: --registry-qps="5"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665664    2580 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665667    2580 flags.go:64] FLAG: --reserved-memory=""
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665671    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 03:27:40.672660 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665674    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665677    2580 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665679    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665682    2580 flags.go:64] FLAG: --runonce="false"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665685    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665688    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665691    2580 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665694    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665696    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665699    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665703    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665706    2580 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665708    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665711    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665714    2580 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665725    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665733    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665736    2580 flags.go:64] FLAG: --system-cgroups=""
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665739    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665744    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665747    2580 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665750    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665758    2580 flags.go:64] FLAG: --tls-min-version=""
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665760    2580 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665764    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 03:27:40.673304 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665767    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665770    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665773    2580 flags.go:64] FLAG: --v="2"
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665778    2580 flags.go:64] FLAG: --version="false"
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665782    2580 flags.go:64] FLAG: --vmodule=""
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665785    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.665789    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.665963    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.665968    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.665971    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.665974    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.665977    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.665979    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.665982    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.665985    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.665987    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.665989    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.665992    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.665996    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.665999    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:40.673898 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666002    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666005    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666008    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666011    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666019    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666022    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666025    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666027    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666030    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666034    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666037    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666040    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666043    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666045    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666048    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666050    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666053    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666055    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666058    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666060    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:40.674435 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666063    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666065    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666068    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666070    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666073    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666075    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666077    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666080    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666083    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666085    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666087    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666090    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666093    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666095    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666098    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666100    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666103    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666111    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666114    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:40.674931 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666117    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666119    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666123    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666125    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666128    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666131    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666133    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666136    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666138    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666141    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666143    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666145    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666148    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666151    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666153    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666156    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666158    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666161    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666163    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666165    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:40.675454 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666168    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666170    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666172    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666175    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666177    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666180    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666182    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666184    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666187    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666189    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666191    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666199    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666202    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666204    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666211    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666213    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666217    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666220    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666222    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666224    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:40.676213 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666227    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666229    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666232    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666240    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666242    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666245    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666247    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666250    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666252    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666255    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666257    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666260    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666262    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666281    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.666312    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666361    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:40.676998 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666365    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666369    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666372    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666374    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666377    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666380    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666382    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666385    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666393    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666396    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666399    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666403    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666407    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666409    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666412    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666414    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666417    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666420    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666422    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:40.677517 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666424    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666427    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666429    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666432    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666435    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666437    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666439    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666442    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666444    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666447    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666450    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666452    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666455    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666457    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666459    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666462    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666464    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666467    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666469    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666471    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:40.677992 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666474    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666477    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666485    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666487    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666491    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666495    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666497    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666500    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666502    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666504    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666507    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666509    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666512    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666514    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666517    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666519    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666521    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666524    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666527    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666529    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:40.678609 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666532    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666534    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666538    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666540    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666543    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666545    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666548    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666550    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666553    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666555    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666559    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666563    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666565    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666568    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666571    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666579    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666583    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666587    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666589    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:40.679083 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666592    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666594    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666598    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666602    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666605    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666608    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666610    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666613    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666616    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666618    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666621    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666623    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666625    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666635    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666639    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666643    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666645    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666648    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666650    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:40.679558 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666653    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666655    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666658    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666660    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666662    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666665    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666667    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.666670    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.674110    2580 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.674129    2580 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.674797    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.674804    2580 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674893    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674898    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674901    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674904    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674906    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674909    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674912    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674915    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:40.680042 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674917    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674920    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674923    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674926    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674928    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674932    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674935    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674937    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674940    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674944    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674948    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674951    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674954    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674956    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674959    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674962    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674964    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674967    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674969    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:40.680619 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674971    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674974    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674977    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674990    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674992    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674995    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.674997    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675000    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675003    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675005    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675008    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675010    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675013    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675015    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675018    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675020    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675023    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675026    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675028    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:40.681079 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675030    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675033    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675036    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675038    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675041    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675043    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675046    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675048    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675052    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675056    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675058    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675061    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675064    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675066    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675069    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675072    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675080    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675089    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675092    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:40.681640 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675094    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675097    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675099    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675102    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675104    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675107    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675109    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675111    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675114    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675117    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675119    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675122    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675124    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675126    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675129    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675132    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675134    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675137    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675139    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675141    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:40.682262 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675144    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675146    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675149    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675151    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675153    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675156    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675158    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675161    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675164    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675166    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675169    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675177    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675180    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675182    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675184    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675187    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675189    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675192    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675194    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675196    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:40.683089 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.675199    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:40.683633 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.675227    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 03:27:40.683633 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676142    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:40.683633 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676175    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:40.683633 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676181    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:40.683633 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676186    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:40.683633 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676191    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:40.683633 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676195    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:40.683633 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676199    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:40.683633 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676212    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:40.683633 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676219    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:40.683633 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676224    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:40.683633 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676228    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:40.683633 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676232    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:40.683633 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676237    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:40.683633 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676241    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676247    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676253    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676258    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676262    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676283    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676293    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676297    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676301    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676306    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676310    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676314    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676319    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676323    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676327    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676332    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676336    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676380    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676385    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676395    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:40.684119 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676429    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676434    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676438    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676442    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676446    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676451    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676454    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676459    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676463    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676467    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676471    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676476    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676485    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676489    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676494    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676499    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676503    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676507    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676512    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:40.684774 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676516    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676520    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676525    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676529    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676534    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676543    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676547    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676552    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676556    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676561    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676565    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676569    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676573    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676578    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676612    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676616    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676620    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676625    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676634    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676638    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:40.685237 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676642    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676646    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676649    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676652    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676655    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676657    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676660    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676663    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676666    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676668    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676671    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676676    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676680    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676684    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676689    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676693    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676698    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676702    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676706    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676710    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:40.685741 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676715    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:40.686234 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676719    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:40.686234 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676724    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:40.686234 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676733    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:40.686234 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676737    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:40.686234 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676741    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:40.686234 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676745    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:40.686234 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676779    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:40.686234 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676783    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:40.686234 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676788    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:40.686234 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676792    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:40.686234 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676796    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:40.686234 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676800    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:40.686234 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.676805    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:40.686234 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.677031    2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 03:27:40.686234 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677085    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677091    2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677095    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677099    2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677103    2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677107    2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677110    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677113    2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677115    2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677118    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677120    2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677123    2580 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677126    2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677128    2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677131    2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677134    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677136    2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677139    2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677142    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677144    2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:27:40.686611 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677148    2580 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677151    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677153    2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677156    2580 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677158    2580 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677161    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677164    2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677166    2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677169    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677171    2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677174    2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677176    2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677179    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677182    2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677184    2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677187    2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677190    2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677193    2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677195    2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677198    2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:27:40.687088 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677201    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677203    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677206    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677208    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677211    2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677213    2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677216    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677219    2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677221    2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677224    2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677226    2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677229    2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677231    2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677234    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677236    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677239    2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677244    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677247    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677249    2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:27:40.687564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677252    2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677255    2580 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677260    2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677282    2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677286    2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677290    2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677293    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677296    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677300    2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677304    2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677308    2580 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677311    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677313    2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677316    2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677319    2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677321    2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677325    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677328    2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677331    2580 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:27:40.688015 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677333    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677336    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677338    2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677340    2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677343    2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677346    2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677348    2580 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677351    2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677353    2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677356    2580 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677360    2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677362    2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677365    2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677367    2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677370    2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677372    2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677375    2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677377    2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677379    2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:27:40.688545 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677382    2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:27:40.688993 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677385    2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:27:40.688993 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677388    2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:27:40.688993 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677390    2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:27:40.688993 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677393    2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:27:40.688993 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677396    2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:27:40.688993 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677399    2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:27:40.688993 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677403    2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:27:40.688993 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:40.677405    2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:27:40.688993 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.677477    2580 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 03:27:40.688993 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.679555    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 31 03:27:40.688993 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.680427    2580 server.go:1014] "Starting client certificate rotation"
Oct 31 03:27:40.688993 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.680534    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 03:27:40.688993 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.681177    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 03:27:40.688993 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.682325    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 03:27:40.688993 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.683518    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 03:27:40.696327 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.696310    2580 log.go:25] "Validated CRI v1 runtime API"
Oct 31 03:27:40.699427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.699414    2580 log.go:25] "Validated CRI v1 image API"
Oct 31 03:27:40.700330 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.700305    2580 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 03:27:40.703536 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.703519    2580 fs.go:135] Filesystem UUIDs: map[50b1979b-bde4-46c8-94d6-889a96d173ee:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 fc3b5342-58de-4686-9cc0-de8f5220e968:/dev/nvme0n1p4]
Oct 31 03:27:40.703604 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.703537    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 31 03:27:40.709078 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.708964    2580 manager.go:217] Machine: {Timestamp:2026-10-31 03:27:40.707627889 +0000 UTC m=+0.325529301 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100312 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f00b6779b2a329a67f9893d000f1d SystemUUID:ec2f00b6-779b-2a32-9a67-f9893d000f1d BootID:b5920995-388f-4b1b-9ff3-a7ada365d64b Filesystems:[{Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:d7:57:ce:25 Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:d7:57:ce:25 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ea:a0:14:52:5b:4f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 03:27:40.709078 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.709077    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 03:27:40.709180 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.709155    2580 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 03:27:40.712027 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.711996    2580 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 03:27:40.712153 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.712029    2580 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-19.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 03:27:40.712199 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.712155    2580 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 03:27:40.712199 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.712164    2580 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 03:27:40.712253 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.712200    2580 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 03:27:40.712253 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.712219    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 03:27:40.713063 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.713053    2580 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 03:27:40.716042 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.716030    2580 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 03:27:40.716154 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.716146    2580 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 03:27:40.716818 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.716809    2580 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 03:27:40.716852 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.716822    2580 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 03:27:40.716852 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.716833    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 03:27:40.716852 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.716841    2580 kubelet.go:405] "Adding apiserver pod source"
Oct 31 03:27:40.716852 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.716849    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 03:27:40.717206 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.717193    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 03:27:40.720971 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.720945    2580 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 03:27:40.725037 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.725023    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 03:27:40.725772 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.725760    2580 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 03:27:40.725813 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.725791    2580 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 03:27:40.726439 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.726427    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 03:27:40.726474 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.726447    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 03:27:40.726474 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.726454    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 03:27:40.726474 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.726460    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 03:27:40.726474 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.726465    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 03:27:40.726474 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.726470    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 03:27:40.727019 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.727011    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 03:27:40.727049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.727020    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 03:27:40.727049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.727027    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 03:27:40.727049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.727032    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 03:27:40.727049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.727040    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 03:27:40.727049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.727050    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 03:27:40.727608 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.727567    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 03:27:40.727608 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.727583    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 03:27:40.731251 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.731237    2580 server.go:1263] "Started kubelet"
Oct 31 03:27:40.731357 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.731319    2580 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 03:27:40.731467 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.731420    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 03:27:40.731516 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.731492    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 03:27:40.731983 ip-10-0-130-19 systemd[1]: Started Kubernetes Kubelet.
Oct 31 03:27:40.732693 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.732665    2580 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 03:27:40.733090 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.732971    2580 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 03:27:40.737028 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:40.737009    2580 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 31 03:27:40.738312 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.738296    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 03:27:40.739312 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.739295    2580 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 03:27:40.739415 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:40.739398    2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-19.us-west-2.compute.internal\" not found"
Oct 31 03:27:40.739958 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.739877    2580 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 03:27:40.739958 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.739906    2580 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 03:27:40.739958 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.739881    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 03:27:40.740110 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.739967    2580 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 03:27:40.740110 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.739975    2580 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 03:27:40.740110 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.740097    2580 factory.go:55] Registering systemd factory
Oct 31 03:27:40.740217 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.740155    2580 factory.go:223] Registration of the systemd container factory successfully
Oct 31 03:27:40.740512 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.740499    2580 factory.go:153] Registering CRI-O factory
Oct 31 03:27:40.740574 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.740515    2580 factory.go:223] Registration of the crio container factory successfully
Oct 31 03:27:40.740574 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.740563    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 03:27:40.740675 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.740580    2580 factory.go:103] Registering Raw factory
Oct 31 03:27:40.740675 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.740599    2580 manager.go:1199] Started watching for new ooms in manager
Oct 31 03:27:40.741469 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.741454    2580 manager.go:319] Starting recovery of all containers
Oct 31 03:27:40.753383 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.751414    2580 manager.go:324] Recovery completed
Oct 31 03:27:40.757115 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.757101    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:27:40.758964 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.758948    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:27:40.759015 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.758982    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:27:40.759015 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.758993    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:27:40.759485 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.759471    2580 cpu_manager.go:226] "Starting" policy="none"
Oct 31 03:27:40.759542 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.759485    2580 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 03:27:40.759542 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.759502    2580 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 03:27:40.759598 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.759475    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:27:40.760203 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.760190    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:27:40.760312 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.760214    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:27:40.760312 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.760224    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:27:40.760744 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.760733    2580 policy_none.go:50] "Start"
Oct 31 03:27:40.760778 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.760749    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 03:27:40.760778 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.760759    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 03:27:40.761819 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.761808    2580 policy_none.go:44] "Start"
Oct 31 03:27:40.811877 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.802857    2580 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 03:27:40.811877 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.807212    2580 manager.go:341] "Starting Device Plugin manager"
Oct 31 03:27:40.811877 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:40.807240    2580 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 03:27:40.811877 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.807250    2580 server.go:87] "Starting device plugin registration server"
Oct 31 03:27:40.811877 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.807482    2580 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 03:27:40.811877 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.807491    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 03:27:40.811877 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.807596    2580 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 03:27:40.811877 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.807672    2580 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 03:27:40.811877 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.807679    2580 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 03:27:40.811877 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:40.808252    2580 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 03:27:40.811877 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:40.808364    2580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-19.us-west-2.compute.internal\" not found"
Oct 31 03:27:40.814119 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.813013    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 03:27:40.814530 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.814513    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 03:27:40.814598 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.814536    2580 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 03:27:40.814598 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.814563    2580 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 03:27:40.814678 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:40.814597    2580 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 31 03:27:40.872509 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:40.872461    2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-19.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 31 03:27:40.872509 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.872479    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-19.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 31 03:27:40.873399 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:40.872467    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-19.us-west-2.compute.internal.18e37f7f27f005bd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-19.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-19.us-west-2.compute.internal,},FirstTimestamp:2026-10-31 03:27:40.731205053 +0000 UTC m=+0.349106467,LastTimestamp:2026-10-31 03:27:40.731205053 +0000 UTC m=+0.349106467,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-19.us-west-2.compute.internal,}"
Oct 31 03:27:40.907743 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.907684    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:27:40.908524 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.908508    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:27:40.908618 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.908535    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:27:40.908618 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.908545    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:27:40.908618 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.908569    2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:40.915119 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.915102    2580 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal"]
Oct 31 03:27:40.915207 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.915169    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:27:40.915900 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.915887    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:27:40.915982 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.915916    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:27:40.915982 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.915932    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:27:40.917402 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.917388    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:27:40.917544 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.917530    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:40.917602 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.917558    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:27:40.918082 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.918062    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:27:40.918161 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.918091    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:27:40.918161 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.918110    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:27:40.918161 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.918064    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:27:40.918335 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.918182    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:27:40.918335 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.918199    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:27:40.919612 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.919597    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:40.919695 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.919625    2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:27:40.920383 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.920367    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:27:40.920455 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.920397    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:27:40.920455 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.920407    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:27:40.942100 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.942073    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 03:27:40.942202 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:40.942120    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-btzxn"
Oct 31 03:27:40.949392 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:40.949369    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-19.us-west-2.compute.internal\" not found" node="ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:40.952961 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:40.952943    2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-19.us-west-2.compute.internal\" not found" node="ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:41.025570 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.025541    2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:27:41.027122 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.027106    2580 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:27:41.040814 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.040786    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8cce19b2c56183ccd97ae9b3e0b6ff57-config\") pod \"kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"8cce19b2c56183ccd97ae9b3e0b6ff57\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:41.040923 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.040816    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/af6bcf6162f2d61de6ecd5c68ef93ef6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"af6bcf6162f2d61de6ecd5c68ef93ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:41.040923 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.040845    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af6bcf6162f2d61de6ecd5c68ef93ef6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"af6bcf6162f2d61de6ecd5c68ef93ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:41.121810 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.121780    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:27:41.122982 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.122958    2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:27:41.123389 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.123374    2580 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:41.123436 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:41.123401    2580 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-19.us-west-2.compute.internal\": node \"ip-10-0-130-19.us-west-2.compute.internal\" not found"
Oct 31 03:27:41.139624 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.139607    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:41.141111 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.141095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/af6bcf6162f2d61de6ecd5c68ef93ef6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"af6bcf6162f2d61de6ecd5c68ef93ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:41.141157 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.141121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af6bcf6162f2d61de6ecd5c68ef93ef6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"af6bcf6162f2d61de6ecd5c68ef93ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:41.141157 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.141145    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8cce19b2c56183ccd97ae9b3e0b6ff57-config\") pod \"kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"8cce19b2c56183ccd97ae9b3e0b6ff57\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:41.141223 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.141207    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/af6bcf6162f2d61de6ecd5c68ef93ef6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"af6bcf6162f2d61de6ecd5c68ef93ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:41.141260 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.141208    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8cce19b2c56183ccd97ae9b3e0b6ff57-config\") pod \"kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"8cce19b2c56183ccd97ae9b3e0b6ff57\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:41.141306 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.141208    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af6bcf6162f2d61de6ecd5c68ef93ef6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"af6bcf6162f2d61de6ecd5c68ef93ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:41.248106 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.248082    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 03:27:41.248253 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.248179    2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:41.252211 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.252197    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:41.256452 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.256290    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:27:41.343837 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.343816    2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 03:27:41.680656 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.680573    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 31 03:27:41.681130 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.680802    2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 31 03:27:41.681130 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.680808    2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 31 03:27:41.681130 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.680819    2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 31 03:27:41.681130 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:41.680754    2580 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://aad67609fd0bc481f905aa31524c4d83-10f2963fa6f7f4fa.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-130-19.us-west-2.compute.internal.18e37f7f29979e2a\": read tcp 10.0.130.19:51806->13.184.36.174:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-130-19.us-west-2.compute.internal.18e37f7f29979e2a  default   4823 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-19.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-19.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-19.us-west-2.compute.internal,},FirstTimestamp:2026-10-31 03:27:40 +0000 UTC,LastTimestamp:2026-10-31 03:27:40.915900161 +0000 UTC m=+0.533801590,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-19.us-west-2.compute.internal,}"
Oct 31 03:27:41.717644 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.717609    2580 apiserver.go:52] "Watching apiserver"
Oct 31 03:27:41.738799 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.738769    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 31 03:27:41.946394 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.946297    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 03:22:41 +0000 UTC" deadline="2028-06-02 20:08:34.175982423 +0000 UTC"
Oct 31 03:27:41.946394 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.946339    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13936h40m52.229646317s"
Oct 31 03:27:41.978251 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.978221    2580 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 03:27:41.978571 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.978552    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-wwpg9","kube-system/global-pull-secret-syncer-phb2z","openshift-cluster-node-tuning-operator/tuned-z9phd","openshift-dns/node-resolver-g95zb","openshift-image-registry/node-ca-fd7k8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal","openshift-multus/multus-additional-cni-plugins-xtmjj","openshift-multus/multus-c2qwl","kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4","kube-system/konnectivity-agent-qwh92","openshift-multus/network-metrics-daemon-65ph6","openshift-network-diagnostics/network-check-target-n6jzp","openshift-network-operator/iptables-alerter-zwpfx"]
Oct 31 03:27:41.980038 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.980021    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:41.981285 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.981256    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:41.982525 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.982507    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:41.983590 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.983575    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:27:41.984695 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.984680    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:27:41.986023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.986011    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:41.987184 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.987169    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:41.987290 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:41.987252    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:27:41.988322 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.988312    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:41.988360 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:41.988349    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:27:41.989578 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.989563    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:41.990971 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.990952    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:27:41.992334 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.992319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:41.992419 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:41.992374    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:27:41.993615 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:41.993599    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:27:42.029968 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.029952    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7b9lc\""
Oct 31 03:27:42.030291 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.030243    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 03:27:42.030291 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.030249    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 03:27:42.030291 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.030249    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 03:27:42.030291 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.030283    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 03:27:42.030473 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.030340    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 03:27:42.030805 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.030793    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 03:27:42.031140 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.031125    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 03:27:42.031230 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.031223    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hp5ds\""
Oct 31 03:27:42.031331 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.031222    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 03:27:42.031384 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.031353    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 03:27:42.031433 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.031414    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 03:27:42.031478 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.031452    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 03:27:42.031863 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.031850    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zvcgd\""
Oct 31 03:27:42.031966 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.031952    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:27:42.032014 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.031955    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 03:27:42.040955 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.040940    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 03:27:42.044422 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0883de4-ab87-4a25-bd8b-ecd4914d4638-konnectivity-ca\") pod \"konnectivity-agent-qwh92\" (UID: \"a0883de4-ab87-4a25-bd8b-ecd4914d4638\") " pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:27:42.044478 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f03b006-c2d0-4e6b-920b-88d146280084-host\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:27:42.044478 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044446    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-kubelet\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.044478 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-etc-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.044478 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysctl-d\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.044610 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysctl-conf\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.044610 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044533    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:42.044610 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044568    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-env-overrides\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.044610 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044602    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-kubernetes\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.044723 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044625    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-sys-fs\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.044723 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-hostroot\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.044723 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.044723 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044687    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cnibin\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.044723 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044713    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.044863 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044737    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-systemd-units\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.044863 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044756    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:42.044863 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62xwv\" (UniqueName: \"kubernetes.io/projected/d78ab9e2-f553-441e-9f3d-6d8985085c89-kube-api-access-62xwv\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:42.044863 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s87l5\" (UniqueName: \"kubernetes.io/projected/694e1b6d-1615-4134-b262-e9df58af99b5-kube-api-access-s87l5\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:27:42.044863 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044850    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-dbus\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:42.045029 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044872    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-registration-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.045029 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm2wv\" (UniqueName: \"kubernetes.io/projected/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-kube-api-access-zm2wv\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.045029 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044901    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-host\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.045029 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044920    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-conf-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.045029 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.044968    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56fwn\" (UniqueName: \"kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn\") pod \"network-check-target-n6jzp\" (UID: \"05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8\") " pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:42.045029 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045010    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-etc-selinux\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.045216 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045033    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfb57\" (UniqueName: \"kubernetes.io/projected/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-kube-api-access-rfb57\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.045216 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovn-node-metrics-cert\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.045216 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045071    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysconfig\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.045216 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045093    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spbdb\" (UniqueName: \"kubernetes.io/projected/7ccdcc49-f072-4e15-b021-9bffc3fb674a-kube-api-access-spbdb\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.045216 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-socket-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.045216 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045135    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7svn9\" (UniqueName: \"kubernetes.io/projected/4700dca5-f15a-4329-b719-828f5475810e-kube-api-access-7svn9\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:27:42.045216 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045165    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-cni-bin\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.045216 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-lib-modules\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.045216 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045204    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ztgb\" (UniqueName: \"kubernetes.io/projected/65e4e478-1f58-498d-b6eb-619307c2943f-kube-api-access-6ztgb\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.045558 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-cnibin\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.045558 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045249    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-etc-kubernetes\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.045558 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-run-ovn-kubernetes\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.045558 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045315    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-netns\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.045558 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045334    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-cni-bin\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.045558 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f03b006-c2d0-4e6b-920b-88d146280084-serviceca\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:27:42.045558 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045370    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-kubelet-config\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:42.045558 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045413    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-cni-multus\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.045558 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045442    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.045558 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045468    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-log-socket\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.045558 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.045558 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045504    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-modprobe-d\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.045558 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045518    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-run\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.045558 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045539    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ccdcc49-f072-4e15-b021-9bffc3fb674a-cni-binary-copy\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.045558 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045565    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-slash\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045586    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-systemd\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045606    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65e4e478-1f58-498d-b6eb-619307c2943f-tmp\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-kubelet\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045643    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-run-netns\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045657    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-cni-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045675    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-daemon-config\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-device-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-node-log\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045834    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgprc\" (UniqueName: \"kubernetes.io/projected/3add72f4-7034-4186-95b5-59d2bd770827-kube-api-access-tgprc\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045857    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-systemd\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045894    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-var-lib-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045917    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovnkube-script-lib\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045955    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-k8s-cni-cncf-io\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.046193 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.045979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0883de4-ab87-4a25-bd8b-ecd4914d4638-agent-certs\") pod \"konnectivity-agent-qwh92\" (UID: \"a0883de4-ab87-4a25-bd8b-ecd4914d4638\") " pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/694e1b6d-1615-4134-b262-e9df58af99b5-hosts-file\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046034    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-cni-netd\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jdm6\" (UniqueName: \"kubernetes.io/projected/5f03b006-c2d0-4e6b-920b-88d146280084-kube-api-access-7jdm6\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4700dca5-f15a-4329-b719-828f5475810e-host-slash\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046119    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-sys\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046138    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/65e4e478-1f58-498d-b6eb-619307c2943f-etc-tuned\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046166    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-socket-dir-parent\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046201    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-multus-certs\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046224    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/694e1b6d-1615-4134-b262-e9df58af99b5-tmp-dir\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046285    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4700dca5-f15a-4329-b719-828f5475810e-iptables-alerter-script\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046317    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-system-cni-dir\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046340    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-ovn\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovnkube-config\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046384    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-var-lib-kubelet\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.046738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.046414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-system-cni-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.075094 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.075077    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 03:27:42.075094 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.075078    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 03:27:42.075292 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.075261    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 03:27:42.075353 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.075298    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hztlx\""
Oct 31 03:27:42.075353 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.075296    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 03:27:42.077294 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.077279    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 03:27:42.077354 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.077338    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 03:27:42.077397 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.077357    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z9sxx\""
Oct 31 03:27:42.077397 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.077374    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-dlcmg\""
Oct 31 03:27:42.077775 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.077761    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 03:27:42.077863 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.077850    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hp8t5\""
Oct 31 03:27:42.078882 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.078864    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 03:27:42.078949 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.078896    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 03:27:42.078949 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.078911    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 03:27:42.078949 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.078921    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 03:27:42.078949 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.078925    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8nlcv\""
Oct 31 03:27:42.078949 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.078899    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2rc66\""
Oct 31 03:27:42.079197 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.078941    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 03:27:42.079197 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.078987    2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 31 03:27:42.079197 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.078922    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:27:42.079197 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.078943    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 03:27:42.079197 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.079041    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w952m"
Oct 31 03:27:42.147370 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:42.147458 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147375    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-env-overrides\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.147458 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-kubernetes\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.147458 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-sys-fs\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.147458 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-hostroot\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.147458 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.147700 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cnibin\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.147700 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147501    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-hostroot\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.147700 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147512    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-sys-fs\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.147700 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147520    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.147700 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.147504    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:42.147700 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147537    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.147700 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147544    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cnibin\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.147700 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147493    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-kubernetes\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.147700 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147570    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-systemd-units\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.147700 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.147607    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret podName:609f4320-8b1b-459e-9e9b-9aeeed31dbe2 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:42.647572001 +0000 UTC m=+2.265473401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret") pod "global-pull-secret-syncer-phb2z" (UID: "609f4320-8b1b-459e-9e9b-9aeeed31dbe2") : object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:42.147700 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147615    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-systemd-units\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.147700 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:42.147700 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62xwv\" (UniqueName: \"kubernetes.io/projected/d78ab9e2-f553-441e-9f3d-6d8985085c89-kube-api-access-62xwv\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:42.147700 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147673    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s87l5\" (UniqueName: \"kubernetes.io/projected/694e1b6d-1615-4134-b262-e9df58af99b5-kube-api-access-s87l5\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:27:42.147700 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.147696    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-dbus\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.147742    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs podName:d78ab9e2-f553-441e-9f3d-6d8985085c89 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:42.647724034 +0000 UTC m=+2.265625452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs") pod "network-metrics-daemon-65ph6" (UID: "d78ab9e2-f553-441e-9f3d-6d8985085c89") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147770    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-registration-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm2wv\" (UniqueName: \"kubernetes.io/projected/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-kube-api-access-zm2wv\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147851    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-registration-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147859    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-env-overrides\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-host\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147895    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-dbus\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-conf-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147941    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-conf-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147946    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-host\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147949    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147965    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56fwn\" (UniqueName: \"kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn\") pod \"network-check-target-n6jzp\" (UID: \"05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8\") " pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.147987    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-etc-selinux\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfb57\" (UniqueName: \"kubernetes.io/projected/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-kube-api-access-rfb57\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.148427 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovn-node-metrics-cert\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148039    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysconfig\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148060    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spbdb\" (UniqueName: \"kubernetes.io/projected/7ccdcc49-f072-4e15-b021-9bffc3fb674a-kube-api-access-spbdb\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148083    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-socket-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148095    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-etc-selinux\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7svn9\" (UniqueName: \"kubernetes.io/projected/4700dca5-f15a-4329-b719-828f5475810e-kube-api-access-7svn9\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148116    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysconfig\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-cni-bin\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-lib-modules\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ztgb\" (UniqueName: \"kubernetes.io/projected/65e4e478-1f58-498d-b6eb-619307c2943f-kube-api-access-6ztgb\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-cnibin\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148226    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-socket-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-etc-kubernetes\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148286    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-run-ovn-kubernetes\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148292    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-cnibin\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148287    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-etc-kubernetes\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148305    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-lib-modules\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148228    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-cni-bin\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.149166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148321    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-netns\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148335    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-run-ovn-kubernetes\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148350    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-netns\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148360    2580 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148385    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-cni-bin\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148412    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-cni-bin\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f03b006-c2d0-4e6b-920b-88d146280084-serviceca\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-kubelet-config\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-cni-multus\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148496    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-log-socket\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148539    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-cni-multus\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148577    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-modprobe-d\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148524    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-kubelet-config\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148583    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-log-socket\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148601    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-run\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148618    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.150031 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ccdcc49-f072-4e15-b021-9bffc3fb674a-cni-binary-copy\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148661    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-run\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148678    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-slash\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-systemd\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148710    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-modprobe-d\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65e4e478-1f58-498d-b6eb-619307c2943f-tmp\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148750    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-slash\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-kubelet\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148784    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-kubelet\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-run-netns\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148829    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-run-netns\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148892    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-systemd\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148925    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-cni-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-daemon-config\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-device-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148991    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-cni-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.148998    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149019    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.151049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-node-log\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgprc\" (UniqueName: \"kubernetes.io/projected/3add72f4-7034-4186-95b5-59d2bd770827-kube-api-access-tgprc\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149084    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149109    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-systemd\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-var-lib-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovnkube-script-lib\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-k8s-cni-cncf-io\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149206    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0883de4-ab87-4a25-bd8b-ecd4914d4638-agent-certs\") pod \"konnectivity-agent-qwh92\" (UID: \"a0883de4-ab87-4a25-bd8b-ecd4914d4638\") " pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/694e1b6d-1615-4134-b262-e9df58af99b5-hosts-file\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-cni-netd\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jdm6\" (UniqueName: \"kubernetes.io/projected/5f03b006-c2d0-4e6b-920b-88d146280084-kube-api-access-7jdm6\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149322    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4700dca5-f15a-4329-b719-828f5475810e-host-slash\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-sys\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/65e4e478-1f58-498d-b6eb-619307c2943f-etc-tuned\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149446    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-socket-dir-parent\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149470    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-multus-certs\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.151950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149478    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/694e1b6d-1615-4134-b262-e9df58af99b5-tmp-dir\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4700dca5-f15a-4329-b719-828f5475810e-iptables-alerter-script\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149552    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-system-cni-dir\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149576    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-ovn\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovnkube-config\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149619    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f03b006-c2d0-4e6b-920b-88d146280084-serviceca\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149626    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-var-lib-kubelet\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-system-cni-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149734    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-var-lib-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149744    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-systemd\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149752    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ccdcc49-f072-4e15-b021-9bffc3fb674a-cni-binary-copy\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0883de4-ab87-4a25-bd8b-ecd4914d4638-konnectivity-ca\") pod \"konnectivity-agent-qwh92\" (UID: \"a0883de4-ab87-4a25-bd8b-ecd4914d4638\") " pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149807    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f03b006-c2d0-4e6b-920b-88d146280084-host\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-kubelet\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149856    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-etc-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149879    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysctl-d\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.149903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysctl-conf\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.152494 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150065    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysctl-conf\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150101    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-daemon-config\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150108    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-device-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150143    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-node-log\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150183    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-system-cni-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150213    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/694e1b6d-1615-4134-b262-e9df58af99b5-hosts-file\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150233    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovnkube-script-lib\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150299    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4700dca5-f15a-4329-b719-828f5475810e-iptables-alerter-script\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150309    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-k8s-cni-cncf-io\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150322    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-cni-netd\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150322    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150359    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-socket-dir-parent\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150402    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-multus-certs\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150489    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4700dca5-f15a-4329-b719-828f5475810e-host-slash\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150530    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f03b006-c2d0-4e6b-920b-88d146280084-host\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150566    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-sys\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150570    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-etc-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150592    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-kubelet\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.153023 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150671    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/694e1b6d-1615-4134-b262-e9df58af99b5-tmp-dir\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:27:42.153521 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150675    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysctl-d\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.153521 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150707    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-system-cni-dir\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.153521 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.150739    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-ovn\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.153521 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.151112    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.153521 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.151168    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-var-lib-kubelet\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.153521 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.151198    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovnkube-config\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.153521 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.151198    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0883de4-ab87-4a25-bd8b-ecd4914d4638-konnectivity-ca\") pod \"konnectivity-agent-qwh92\" (UID: \"a0883de4-ab87-4a25-bd8b-ecd4914d4638\") " pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:27:42.153521 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.151889    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65e4e478-1f58-498d-b6eb-619307c2943f-tmp\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.153521 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.151891    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovn-node-metrics-cert\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.153521 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.152036    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/65e4e478-1f58-498d-b6eb-619307c2943f-etc-tuned\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.153521 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.153060    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0883de4-ab87-4a25-bd8b-ecd4914d4638-agent-certs\") pod \"konnectivity-agent-qwh92\" (UID: \"a0883de4-ab87-4a25-bd8b-ecd4914d4638\") " pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:27:42.204475 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.204442    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 03:27:42.204475 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.204476    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 03:27:42.204659 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.204490    2580 projected.go:196] Error preparing data for projected volume kube-api-access-56fwn for pod openshift-network-diagnostics/network-check-target-n6jzp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:42.204659 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.204557    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn podName:05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:42.704537178 +0000 UTC m=+2.322438606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-56fwn" (UniqueName: "kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn") pod "network-check-target-n6jzp" (UID: "05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:42.207365 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.207331    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm2wv\" (UniqueName: \"kubernetes.io/projected/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-kube-api-access-zm2wv\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.207471 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.207338    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfb57\" (UniqueName: \"kubernetes.io/projected/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-kube-api-access-rfb57\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.207471 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.207433    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62xwv\" (UniqueName: \"kubernetes.io/projected/d78ab9e2-f553-441e-9f3d-6d8985085c89-kube-api-access-62xwv\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:42.207594 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.207486    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ztgb\" (UniqueName: \"kubernetes.io/projected/65e4e478-1f58-498d-b6eb-619307c2943f-kube-api-access-6ztgb\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.207594 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.207554    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7svn9\" (UniqueName: \"kubernetes.io/projected/4700dca5-f15a-4329-b719-828f5475810e-kube-api-access-7svn9\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:27:42.207689 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.207672    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s87l5\" (UniqueName: \"kubernetes.io/projected/694e1b6d-1615-4134-b262-e9df58af99b5-kube-api-access-s87l5\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:27:42.208535 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.208517    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgprc\" (UniqueName: \"kubernetes.io/projected/3add72f4-7034-4186-95b5-59d2bd770827-kube-api-access-tgprc\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.208943 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.208927    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spbdb\" (UniqueName: \"kubernetes.io/projected/7ccdcc49-f072-4e15-b021-9bffc3fb674a-kube-api-access-spbdb\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.209537 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.209520    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jdm6\" (UniqueName: \"kubernetes.io/projected/5f03b006-c2d0-4e6b-920b-88d146280084-kube-api-access-7jdm6\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:27:42.320927 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.320884    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:27:42.345056 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.345030    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:27:42.357506 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.357477    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:27:42.357855 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:42.357814    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac8c7ff3_c156_4201_a297_ea6d1fd48e55.slice/crio-ceba6febe347efdf3b0c58e1f0f45026529f478f8944770af62d2b507e27873e WatchSource:0}: Error finding container ceba6febe347efdf3b0c58e1f0f45026529f478f8944770af62d2b507e27873e: Status 404 returned error can't find the container with id ceba6febe347efdf3b0c58e1f0f45026529f478f8944770af62d2b507e27873e
Oct 31 03:27:42.358437 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:42.358416    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6bcf6162f2d61de6ecd5c68ef93ef6.slice/crio-e820823f0827d4209635673a4e355c31477a5ed86dc6d50ad571e6ce46a7f151 WatchSource:0}: Error finding container e820823f0827d4209635673a4e355c31477a5ed86dc6d50ad571e6ce46a7f151: Status 404 returned error can't find the container with id e820823f0827d4209635673a4e355c31477a5ed86dc6d50ad571e6ce46a7f151
Oct 31 03:27:42.359408 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:42.359391    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cce19b2c56183ccd97ae9b3e0b6ff57.slice/crio-e096f86fb472ca5958f5b1079e2a53a7423eedc1a37fdc1c2df72f9989337a96 WatchSource:0}: Error finding container e096f86fb472ca5958f5b1079e2a53a7423eedc1a37fdc1c2df72f9989337a96: Status 404 returned error can't find the container with id e096f86fb472ca5958f5b1079e2a53a7423eedc1a37fdc1c2df72f9989337a96
Oct 31 03:27:42.366565 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.366549    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 03:27:42.367503 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:42.367466    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e4e478_1f58_498d_b6eb_619307c2943f.slice/crio-b9633eed85abe9ec747744da11f68772f9c2ee71c3605f95b2c9d667c192ef88 WatchSource:0}: Error finding container b9633eed85abe9ec747744da11f68772f9c2ee71c3605f95b2c9d667c192ef88: Status 404 returned error can't find the container with id b9633eed85abe9ec747744da11f68772f9c2ee71c3605f95b2c9d667c192ef88
Oct 31 03:27:42.371154 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.371139    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:27:42.378562 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:42.378544    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod694e1b6d_1615_4134_b262_e9df58af99b5.slice/crio-cf51b597e41d1cde6a03e33a56f068a39681f62798cb4a7a6e2ff78d8cd63f94 WatchSource:0}: Error finding container cf51b597e41d1cde6a03e33a56f068a39681f62798cb4a7a6e2ff78d8cd63f94: Status 404 returned error can't find the container with id cf51b597e41d1cde6a03e33a56f068a39681f62798cb4a7a6e2ff78d8cd63f94
Oct 31 03:27:42.389501 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.389482    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:27:42.396393 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:42.396372    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f03b006_c2d0_4e6b_920b_88d146280084.slice/crio-787c619e13f321ed8e26d02ae51fbcb9fefa5c47dae7feecddb0297c6cf5dc88 WatchSource:0}: Error finding container 787c619e13f321ed8e26d02ae51fbcb9fefa5c47dae7feecddb0297c6cf5dc88: Status 404 returned error can't find the container with id 787c619e13f321ed8e26d02ae51fbcb9fefa5c47dae7feecddb0297c6cf5dc88
Oct 31 03:27:42.407508 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.407491    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-c2qwl"
Oct 31 03:27:42.413897 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:42.413876    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ccdcc49_f072_4e15_b021_9bffc3fb674a.slice/crio-08de644521b27389f620f4e962dfa265629780ee93c9c43a23b0a689a3862814 WatchSource:0}: Error finding container 08de644521b27389f620f4e962dfa265629780ee93c9c43a23b0a689a3862814: Status 404 returned error can't find the container with id 08de644521b27389f620f4e962dfa265629780ee93c9c43a23b0a689a3862814
Oct 31 03:27:42.427101 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.427083    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:27:42.434584 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:42.434565    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3add72f4_7034_4186_95b5_59d2bd770827.slice/crio-f3bb901b1e16080db3a21651bacc8711afaf29713df03fc9282ba14ab4583c77 WatchSource:0}: Error finding container f3bb901b1e16080db3a21651bacc8711afaf29713df03fc9282ba14ab4583c77: Status 404 returned error can't find the container with id f3bb901b1e16080db3a21651bacc8711afaf29713df03fc9282ba14ab4583c77
Oct 31 03:27:42.439738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.439719    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:27:42.445419 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.445403    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:27:42.447138 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:42.447095    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0883de4_ab87_4a25_bd8b_ecd4914d4638.slice/crio-a9b81ffeca9738d9b69e850c3c29b774f5e3c7b7923e97044f0388e1832daa88 WatchSource:0}: Error finding container a9b81ffeca9738d9b69e850c3c29b774f5e3c7b7923e97044f0388e1832daa88: Status 404 returned error can't find the container with id a9b81ffeca9738d9b69e850c3c29b774f5e3c7b7923e97044f0388e1832daa88
Oct 31 03:27:42.452342 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:42.452324    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4700dca5_f15a_4329_b719_828f5475810e.slice/crio-bc16d1c6f915a7bd3ac441e8c26b54e6c810a20251dfb14a7f53247f959ddb00 WatchSource:0}: Error finding container bc16d1c6f915a7bd3ac441e8c26b54e6c810a20251dfb14a7f53247f959ddb00: Status 404 returned error can't find the container with id bc16d1c6f915a7bd3ac441e8c26b54e6c810a20251dfb14a7f53247f959ddb00
Oct 31 03:27:42.652430 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.652404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:42.652602 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.652440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:42.652602 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.652557    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:42.652602 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.652557    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:42.652715 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.652608    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs podName:d78ab9e2-f553-441e-9f3d-6d8985085c89 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:43.652593381 +0000 UTC m=+3.270494783 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs") pod "network-metrics-daemon-65ph6" (UID: "d78ab9e2-f553-441e-9f3d-6d8985085c89") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:42.652715 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.652621    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret podName:609f4320-8b1b-459e-9e9b-9aeeed31dbe2 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:43.652614465 +0000 UTC m=+3.270515867 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret") pod "global-pull-secret-syncer-phb2z" (UID: "609f4320-8b1b-459e-9e9b-9aeeed31dbe2") : object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:42.753501 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.753430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56fwn\" (UniqueName: \"kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn\") pod \"network-check-target-n6jzp\" (UID: \"05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8\") " pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:42.754069 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.753549    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 03:27:42.754069 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.753567    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 03:27:42.754069 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.753578    2580 projected.go:196] Error preparing data for projected volume kube-api-access-56fwn for pod openshift-network-diagnostics/network-check-target-n6jzp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:42.754069 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:42.753622    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn podName:05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:43.75361006 +0000 UTC m=+3.371511462 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-56fwn" (UniqueName: "kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn") pod "network-check-target-n6jzp" (UID: "05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:42.819235 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.819174    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal" event={"ID":"af6bcf6162f2d61de6ecd5c68ef93ef6","Type":"ContainerStarted","Data":"e820823f0827d4209635673a4e355c31477a5ed86dc6d50ad571e6ce46a7f151"}
Oct 31 03:27:42.820316 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.820296    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zwpfx" event={"ID":"4700dca5-f15a-4329-b719-828f5475810e","Type":"ContainerStarted","Data":"bc16d1c6f915a7bd3ac441e8c26b54e6c810a20251dfb14a7f53247f959ddb00"}
Oct 31 03:27:42.821415 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.821397    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4" event={"ID":"3add72f4-7034-4186-95b5-59d2bd770827","Type":"ContainerStarted","Data":"f3bb901b1e16080db3a21651bacc8711afaf29713df03fc9282ba14ab4583c77"}
Oct 31 03:27:42.822459 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.822435    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c2qwl" event={"ID":"7ccdcc49-f072-4e15-b021-9bffc3fb674a","Type":"ContainerStarted","Data":"08de644521b27389f620f4e962dfa265629780ee93c9c43a23b0a689a3862814"}
Oct 31 03:27:42.823496 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.823476    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z9phd" event={"ID":"65e4e478-1f58-498d-b6eb-619307c2943f","Type":"ContainerStarted","Data":"b9633eed85abe9ec747744da11f68772f9c2ee71c3605f95b2c9d667c192ef88"}
Oct 31 03:27:42.824447 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.824428    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal" event={"ID":"8cce19b2c56183ccd97ae9b3e0b6ff57","Type":"ContainerStarted","Data":"e096f86fb472ca5958f5b1079e2a53a7423eedc1a37fdc1c2df72f9989337a96"}
Oct 31 03:27:42.825484 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.825466    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qwh92" event={"ID":"a0883de4-ab87-4a25-bd8b-ecd4914d4638","Type":"ContainerStarted","Data":"a9b81ffeca9738d9b69e850c3c29b774f5e3c7b7923e97044f0388e1832daa88"}
Oct 31 03:27:42.826572 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.826552    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fd7k8" event={"ID":"5f03b006-c2d0-4e6b-920b-88d146280084","Type":"ContainerStarted","Data":"787c619e13f321ed8e26d02ae51fbcb9fefa5c47dae7feecddb0297c6cf5dc88"}
Oct 31 03:27:42.827544 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.827526    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g95zb" event={"ID":"694e1b6d-1615-4134-b262-e9df58af99b5","Type":"ContainerStarted","Data":"cf51b597e41d1cde6a03e33a56f068a39681f62798cb4a7a6e2ff78d8cd63f94"}
Oct 31 03:27:42.828556 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:42.828538    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9" event={"ID":"ac8c7ff3-c156-4201-a297-ea6d1fd48e55","Type":"ContainerStarted","Data":"ceba6febe347efdf3b0c58e1f0f45026529f478f8944770af62d2b507e27873e"}
Oct 31 03:27:43.080783 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:43.080678    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 03:22:42 +0000 UTC" deadline="2028-05-01 09:47:05.774680114 +0000 UTC"
Oct 31 03:27:43.080783 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:43.080722    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13158h19m22.693960544s"
Oct 31 03:27:43.083763 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:43.083747    2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:27:43.090848 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:43.090805    2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:27:43.133469 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:43.133444    2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:27:43.284514 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:27:43.284485    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9193a1f4_f4aa_4364_ae16_5520e88d8cc3.slice/crio-355e77d7d042a077b378b74001f86e9ac00bcb2b82e53610cfba4a2b7bdc98b6 WatchSource:0}: Error finding container 355e77d7d042a077b378b74001f86e9ac00bcb2b82e53610cfba4a2b7bdc98b6: Status 404 returned error can't find the container with id 355e77d7d042a077b378b74001f86e9ac00bcb2b82e53610cfba4a2b7bdc98b6
Oct 31 03:27:43.662234 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:43.662198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:43.662442 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:43.662258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:43.662533 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:43.662449    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:43.662533 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:43.662513    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs podName:d78ab9e2-f553-441e-9f3d-6d8985085c89 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:45.662494139 +0000 UTC m=+5.280395558 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs") pod "network-metrics-daemon-65ph6" (UID: "d78ab9e2-f553-441e-9f3d-6d8985085c89") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:43.662653 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:43.662583    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:43.662653 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:43.662628    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret podName:609f4320-8b1b-459e-9e9b-9aeeed31dbe2 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:45.662607002 +0000 UTC m=+5.280508419 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret") pod "global-pull-secret-syncer-phb2z" (UID: "609f4320-8b1b-459e-9e9b-9aeeed31dbe2") : object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:43.763268 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:43.763228    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56fwn\" (UniqueName: \"kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn\") pod \"network-check-target-n6jzp\" (UID: \"05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8\") " pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:43.763917 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:43.763447    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 03:27:43.763917 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:43.763468    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 03:27:43.763917 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:43.763482    2580 projected.go:196] Error preparing data for projected volume kube-api-access-56fwn for pod openshift-network-diagnostics/network-check-target-n6jzp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:43.763917 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:43.763542    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn podName:05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:45.763524204 +0000 UTC m=+5.381425609 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-56fwn" (UniqueName: "kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn") pod "network-check-target-n6jzp" (UID: "05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:43.815627 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:43.815595    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:43.815819 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:43.815734    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:27:43.815890 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:43.815823    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:43.815944 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:43.815895    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:27:43.815987 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:43.815957    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:43.816094 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:43.816051    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:27:43.865098 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:43.865059    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtmjj" event={"ID":"9193a1f4-f4aa-4364-ae16-5520e88d8cc3","Type":"ContainerStarted","Data":"355e77d7d042a077b378b74001f86e9ac00bcb2b82e53610cfba4a2b7bdc98b6"}
Oct 31 03:27:44.082008 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:44.081933    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 03:22:42 +0000 UTC" deadline="2028-06-21 02:11:27.758749136 +0000 UTC"
Oct 31 03:27:44.082008 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:44.081973    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14374h43m43.676781533s"
Oct 31 03:27:45.680681 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:45.680642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:45.681277 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:45.680698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:45.681277 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:45.680809    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:45.681277 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:45.680810    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:45.681277 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:45.680876    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs podName:d78ab9e2-f553-441e-9f3d-6d8985085c89 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:49.680855331 +0000 UTC m=+9.298756751 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs") pod "network-metrics-daemon-65ph6" (UID: "d78ab9e2-f553-441e-9f3d-6d8985085c89") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:45.681277 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:45.680915    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret podName:609f4320-8b1b-459e-9e9b-9aeeed31dbe2 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:49.680896083 +0000 UTC m=+9.298797490 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret") pod "global-pull-secret-syncer-phb2z" (UID: "609f4320-8b1b-459e-9e9b-9aeeed31dbe2") : object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:45.781605 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:45.781566    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56fwn\" (UniqueName: \"kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn\") pod \"network-check-target-n6jzp\" (UID: \"05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8\") " pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:45.781791 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:45.781753    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 03:27:45.781791 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:45.781768    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 03:27:45.781791 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:45.781778    2580 projected.go:196] Error preparing data for projected volume kube-api-access-56fwn for pod openshift-network-diagnostics/network-check-target-n6jzp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:45.781967 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:45.781827    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn podName:05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:49.781814959 +0000 UTC m=+9.399716374 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-56fwn" (UniqueName: "kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn") pod "network-check-target-n6jzp" (UID: "05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:45.815025 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:45.814992    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:45.815281 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:45.815249    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:27:45.815426 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:45.815408    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:45.815529 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:45.815501    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:27:45.815590 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:45.815571    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:45.815957 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:45.815667    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:27:47.815589 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:47.815377    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:47.816053 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:47.815714    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:27:47.816053 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:47.815790    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:47.816053 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:47.815872    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:27:47.816053 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:47.815991    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:47.816242 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:47.816065    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:27:47.900516 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:47.900483    2580 generic.go:356] "Generic (PLEG): container finished" podID="af6bcf6162f2d61de6ecd5c68ef93ef6" containerID="fc0d11997f1713069298ecfd4af1bd8e96fd4bf4b05439a75dc21e15599b3308" exitCode=0
Oct 31 03:27:47.900638 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:47.900572    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal" event={"ID":"af6bcf6162f2d61de6ecd5c68ef93ef6","Type":"ContainerDied","Data":"fc0d11997f1713069298ecfd4af1bd8e96fd4bf4b05439a75dc21e15599b3308"}
Oct 31 03:27:47.910258 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:47.909729    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtmjj" event={"ID":"9193a1f4-f4aa-4364-ae16-5520e88d8cc3","Type":"ContainerStarted","Data":"49575c2e6f379d52c7800996bd05d47610cfb0d4e418f06bda2e7c535e12c360"}
Oct 31 03:27:48.918049 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:48.918002    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qwh92" event={"ID":"a0883de4-ab87-4a25-bd8b-ecd4914d4638","Type":"ContainerStarted","Data":"c1c0a100b1e807b6b132496bd0e2d087328a364827d1e78a74612c7a1fb2f76e"}
Oct 31 03:27:48.924364 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:48.924317    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal" event={"ID":"af6bcf6162f2d61de6ecd5c68ef93ef6","Type":"ContainerStarted","Data":"b5313ea118942dc9041ea29df0cae19b6d4e96492a2929656cde4ad78257a194"}
Oct 31 03:27:48.928307 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:48.928259    2580 generic.go:356] "Generic (PLEG): container finished" podID="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" containerID="49575c2e6f379d52c7800996bd05d47610cfb0d4e418f06bda2e7c535e12c360" exitCode=0
Oct 31 03:27:48.928471 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:48.928382    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtmjj" event={"ID":"9193a1f4-f4aa-4364-ae16-5520e88d8cc3","Type":"ContainerDied","Data":"49575c2e6f379d52c7800996bd05d47610cfb0d4e418f06bda2e7c535e12c360"}
Oct 31 03:27:48.931692 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:48.931665    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal" event={"ID":"8cce19b2c56183ccd97ae9b3e0b6ff57","Type":"ContainerStarted","Data":"d45a9f2c7082837e50c5763363a27bd8114f754e6cf347c4ff877d435482743b"}
Oct 31 03:27:49.158246 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:49.157375    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qwh92" podStartSLOduration=1.51930302 podStartE2EDuration="7.157354612s" totalImagesPullingTime="5.638051592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:27:49.037902019 +0000 UTC m=+8.655803449" watchObservedRunningTime="2026-10-31 03:27:49.157354612 +0000 UTC m=+8.775256039"
Oct 31 03:27:49.274011 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:49.273407    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal" podStartSLOduration=8.273384115 podStartE2EDuration="8.273384115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:27:49.156963368 +0000 UTC m=+8.774864799" watchObservedRunningTime="2026-10-31 03:27:49.273384115 +0000 UTC m=+8.891285542"
Oct 31 03:27:49.274011 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:49.273735    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal" podStartSLOduration=7.273725238 podStartE2EDuration="7.273725238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:27:49.272708975 +0000 UTC m=+8.890610400" watchObservedRunningTime="2026-10-31 03:27:49.273725238 +0000 UTC m=+8.891626664"
Oct 31 03:27:49.718151 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:49.718108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:49.718357 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:49.718170    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:49.718357 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:49.718319    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:49.718482 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:49.718367    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:49.718482 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:49.718413    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret podName:609f4320-8b1b-459e-9e9b-9aeeed31dbe2 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:57.718387852 +0000 UTC m=+17.336289265 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret") pod "global-pull-secret-syncer-phb2z" (UID: "609f4320-8b1b-459e-9e9b-9aeeed31dbe2") : object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:49.718482 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:49.718436    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs podName:d78ab9e2-f553-441e-9f3d-6d8985085c89 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:57.718424922 +0000 UTC m=+17.336326332 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs") pod "network-metrics-daemon-65ph6" (UID: "d78ab9e2-f553-441e-9f3d-6d8985085c89") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:49.815980 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:49.815519    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:49.815980 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:49.815655    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:27:49.815980 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:49.815729    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:49.815980 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:49.815781    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:27:49.815980 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:49.815833    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:49.815980 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:49.815895    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:27:49.819695 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:49.819388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56fwn\" (UniqueName: \"kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn\") pod \"network-check-target-n6jzp\" (UID: \"05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8\") " pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:49.819695 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:49.819541    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 03:27:49.819695 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:49.819554    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 03:27:49.819695 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:49.819564    2580 projected.go:196] Error preparing data for projected volume kube-api-access-56fwn for pod openshift-network-diagnostics/network-check-target-n6jzp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:49.819695 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:49.819601    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn podName:05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8 nodeName:}" failed. No retries permitted until 2026-10-31 03:27:57.819589581 +0000 UTC m=+17.437490983 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-56fwn" (UniqueName: "kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn") pod "network-check-target-n6jzp" (UID: "05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:49.938223 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:49.938135    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4" event={"ID":"3add72f4-7034-4186-95b5-59d2bd770827","Type":"ContainerStarted","Data":"4472b2ed1653351f55f2e452f468b083b51919334bf8cb456364b279e607ec3b"}
Oct 31 03:27:49.942815 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:49.942781    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-fd7k8" event={"ID":"5f03b006-c2d0-4e6b-920b-88d146280084","Type":"ContainerStarted","Data":"1a1d2e9a06cfcf52838fe4a84fe837b8121bf17ae3e1960ca48ae2eef352927b"}
Oct 31 03:27:50.062038 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:50.061707    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-fd7k8" podStartSLOduration=2.216213088 podStartE2EDuration="9.06168717s" totalImagesPullingTime="6.845474082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:27:50.061329536 +0000 UTC m=+9.679230962" watchObservedRunningTime="2026-10-31 03:27:50.06168717 +0000 UTC m=+9.679588599"
Oct 31 03:27:51.102435 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:51.102283    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:27:51.103694 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:51.103595    2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:27:51.815938 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:51.815590    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:51.815938 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:51.815720    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:27:51.815938 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:51.815779    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:51.815938 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:51.815840    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:27:51.815938 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:51.815857    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:51.815938 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:51.815906    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:27:51.949353 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:51.949282    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-c2qwl" event={"ID":"7ccdcc49-f072-4e15-b021-9bffc3fb674a","Type":"ContainerStarted","Data":"70619a04eb895e3578db17db354675098a3d9c831ae578a2074d7ed0830e5393"}
Oct 31 03:27:52.074039 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:52.073095    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-c2qwl" podStartSLOduration=2.051877862 podStartE2EDuration="11.073077628s" totalImagesPullingTime="9.021199766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:27:52.072725613 +0000 UTC m=+11.690627057" watchObservedRunningTime="2026-10-31 03:27:52.073077628 +0000 UTC m=+11.690979055"
Oct 31 03:27:52.154121 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:52.154092    2580 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 03:27:52.824367 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:52.824035    2580 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T03:27:52.154115695Z","UUID":"29419d88-0bbc-45d2-94ec-fa0af8f20496","Handler":null,"Name":"","Endpoint":""}
Oct 31 03:27:52.827041 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:52.827018    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 03:27:52.827163 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:52.827051    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 03:27:52.953532 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:52.953499    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4" event={"ID":"3add72f4-7034-4186-95b5-59d2bd770827","Type":"ContainerStarted","Data":"c4bcb16fac9193eb04b4ee2919d90933dca723d550de9bf1ced6002b51cdd118"}
Oct 31 03:27:52.955564 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:52.955496    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-g95zb" event={"ID":"694e1b6d-1615-4134-b262-e9df58af99b5","Type":"ContainerStarted","Data":"e6fec5f548424642d5cc958d12e3733dbc3db7b98ae1baba1f1f6291b49b4493"}
Oct 31 03:27:53.067169 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:53.066906    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-g95zb" podStartSLOduration=2.222570591 podStartE2EDuration="12.066887743s" totalImagesPullingTime="9.844317152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:27:53.066068547 +0000 UTC m=+12.683969973" watchObservedRunningTime="2026-10-31 03:27:53.066887743 +0000 UTC m=+12.684789166"
Oct 31 03:27:53.815142 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:53.815088    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:53.815611 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:53.815240    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:53.815611 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:53.815285    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:53.815611 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:53.815421    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:27:53.815611 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:53.815461    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:27:53.815611 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:53.815498    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:27:53.959716 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:53.959656    2580 generic.go:356] "Generic (PLEG): container finished" podID="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" containerID="c60712b30d2169117b282d5b021ae4918229007a0fb4ee038d9ae0bc56b6d5df" exitCode=0
Oct 31 03:27:53.959942 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:53.959740    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtmjj" event={"ID":"9193a1f4-f4aa-4364-ae16-5520e88d8cc3","Type":"ContainerDied","Data":"c60712b30d2169117b282d5b021ae4918229007a0fb4ee038d9ae0bc56b6d5df"}
Oct 31 03:27:53.961390 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:53.961363    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zwpfx" event={"ID":"4700dca5-f15a-4329-b719-828f5475810e","Type":"ContainerStarted","Data":"e1b894e5481c9108564b36b6dca11b7a1c54b55dd096ce44d02f5f2b73c4906e"}
Oct 31 03:27:54.185612 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:54.185555    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zwpfx" podStartSLOduration=2.5988419780000003 podStartE2EDuration="12.185542223s" totalImagesPullingTime="9.586700245s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:27:54.185287163 +0000 UTC m=+13.803188577" watchObservedRunningTime="2026-10-31 03:27:54.185542223 +0000 UTC m=+13.803443645"
Oct 31 03:27:54.966162 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:54.966118    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4" event={"ID":"3add72f4-7034-4186-95b5-59d2bd770827","Type":"ContainerStarted","Data":"7ca444057e92ca2ac4062db41c7395365f9ac1cf96a7ed5cfd8e207140a59a91"}
Oct 31 03:27:54.969008 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:54.968982    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z9phd" event={"ID":"65e4e478-1f58-498d-b6eb-619307c2943f","Type":"ContainerStarted","Data":"86eb67c32c3243a26b4c20af79d9b2d61267d1165604ad6f2e7ef572747d8cbf"}
Oct 31 03:27:55.080181 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:55.080119    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4" podStartSLOduration=1.8911309410000001 podStartE2EDuration="13.080100652s" totalImagesPullingTime="11.188969711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:27:55.079279512 +0000 UTC m=+14.697180925" watchObservedRunningTime="2026-10-31 03:27:55.080100652 +0000 UTC m=+14.698002076"
Oct 31 03:27:55.193099 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:55.193033    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-z9phd" podStartSLOduration=2.22113482 podStartE2EDuration="14.19301574s" totalImagesPullingTime="11.97188092s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:27:55.191984612 +0000 UTC m=+14.809886027" watchObservedRunningTime="2026-10-31 03:27:55.19301574 +0000 UTC m=+14.810917163"
Oct 31 03:27:55.814992 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:55.814965    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:55.815086 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:55.815071    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:55.815146 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:55.815087    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:55.815210 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:55.815184    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:27:55.815346 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:55.815232    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:27:55.815404 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:55.815374    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:27:55.973691 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:55.973617    2580 generic.go:356] "Generic (PLEG): container finished" podID="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" containerID="2c920b7d458db552f7de0b86fecae361a303f6e866505f6f944e96800397d71d" exitCode=0
Oct 31 03:27:55.973691 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:55.973654    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtmjj" event={"ID":"9193a1f4-f4aa-4364-ae16-5520e88d8cc3","Type":"ContainerDied","Data":"2c920b7d458db552f7de0b86fecae361a303f6e866505f6f944e96800397d71d"}
Oct 31 03:27:57.783937 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:57.783732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:57.784341 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:57.783974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:57.784341 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:57.783879    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:57.784341 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:57.784077    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret podName:609f4320-8b1b-459e-9e9b-9aeeed31dbe2 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:13.784056253 +0000 UTC m=+33.401957667 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret") pod "global-pull-secret-syncer-phb2z" (UID: "609f4320-8b1b-459e-9e9b-9aeeed31dbe2") : object "kube-system"/"original-pull-secret" not registered
Oct 31 03:27:57.784341 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:57.784089    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:57.784341 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:57.784138    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs podName:d78ab9e2-f553-441e-9f3d-6d8985085c89 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:13.784126507 +0000 UTC m=+33.402027910 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs") pod "network-metrics-daemon-65ph6" (UID: "d78ab9e2-f553-441e-9f3d-6d8985085c89") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 31 03:27:57.815532 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:57.815496    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:57.815683 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:57.815500    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:57.815683 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:57.815622    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:57.815683 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:57.815622    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:27:57.815840 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:57.815713    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:27:57.815840 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:57.815799    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:27:57.884366 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:57.884287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56fwn\" (UniqueName: \"kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn\") pod \"network-check-target-n6jzp\" (UID: \"05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8\") " pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:57.884535 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:57.884439    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 31 03:27:57.884535 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:57.884464    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 31 03:27:57.884535 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:57.884478    2580 projected.go:196] Error preparing data for projected volume kube-api-access-56fwn for pod openshift-network-diagnostics/network-check-target-n6jzp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:57.884669 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:57.884551    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn podName:05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:13.884533137 +0000 UTC m=+33.502434559 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-56fwn" (UniqueName: "kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn") pod "network-check-target-n6jzp" (UID: "05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 31 03:27:57.980864 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:57.980830    2580 generic.go:356] "Generic (PLEG): container finished" podID="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" containerID="d9274d110534008c7b6e283ba2c765030d67a7ddaf8306717390282dc3a00f32" exitCode=0
Oct 31 03:27:57.981038 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:57.980895    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtmjj" event={"ID":"9193a1f4-f4aa-4364-ae16-5520e88d8cc3","Type":"ContainerDied","Data":"d9274d110534008c7b6e283ba2c765030d67a7ddaf8306717390282dc3a00f32"}
Oct 31 03:27:59.815216 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:59.815001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:27:59.815671 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:59.815026    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:27:59.815671 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:59.815325    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:27:59.815671 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:59.815380    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:27:59.815671 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:59.815072    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:27:59.815671 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:27:59.815461    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:27:59.990359 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:59.990329    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9" event={"ID":"ac8c7ff3-c156-4201-a297-ea6d1fd48e55","Type":"ContainerStarted","Data":"556d7c516ec82591fe3b3cf17dac8f961ba463d80f035a72e615d324d626494c"}
Oct 31 03:27:59.990514 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:59.990408    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9" event={"ID":"ac8c7ff3-c156-4201-a297-ea6d1fd48e55","Type":"ContainerStarted","Data":"31ee6fac5942a54caa8e4e3b0f79efc8338f10c858f2701dfc7cc0cb307a94a8"}
Oct 31 03:27:59.990514 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:59.990438    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9" event={"ID":"ac8c7ff3-c156-4201-a297-ea6d1fd48e55","Type":"ContainerStarted","Data":"448c40d523890e887b66e039afa89f55d839a3ee69354a082b1a9d174c99456d"}
Oct 31 03:27:59.990514 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:59.990497    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9" event={"ID":"ac8c7ff3-c156-4201-a297-ea6d1fd48e55","Type":"ContainerStarted","Data":"73726c9b2ff50365968e2fbad3eda98caee567a14d227bdf67c573d90c4b6892"}
Oct 31 03:27:59.990641 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:59.990545    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9" event={"ID":"ac8c7ff3-c156-4201-a297-ea6d1fd48e55","Type":"ContainerStarted","Data":"58896e7448894354a409e92c43978e7dc92cf911e83ae815864ae6d585471441"}
Oct 31 03:27:59.990641 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:27:59.990589    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9" event={"ID":"ac8c7ff3-c156-4201-a297-ea6d1fd48e55","Type":"ContainerStarted","Data":"d656b2acaf5f54061668121a15a74ec8c5cb5903e099fb29a2853c3e08ea322f"}
Oct 31 03:28:00.277342 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:00.277301    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:28:00.277539 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:00.277434    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 03:28:00.278011 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:00.277988    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:28:00.994462 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:00.994425    2580 generic.go:356] "Generic (PLEG): container finished" podID="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" containerID="75ac26d85528a2fb4e37d0ff3c94c309ee97baf25914e946539f00f9a73939d3" exitCode=0
Oct 31 03:28:00.994987 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:00.994463    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtmjj" event={"ID":"9193a1f4-f4aa-4364-ae16-5520e88d8cc3","Type":"ContainerDied","Data":"75ac26d85528a2fb4e37d0ff3c94c309ee97baf25914e946539f00f9a73939d3"}
Oct 31 03:28:01.815509 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:01.815471    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:28:01.815689 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:01.815519    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:01.815689 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:01.815572    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:01.815689 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:01.815588    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:28:01.815689 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:01.815636    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:28:01.815835 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:01.815708    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:28:01.999952 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:01.999914    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9" event={"ID":"ac8c7ff3-c156-4201-a297-ea6d1fd48e55","Type":"ContainerStarted","Data":"c0643626ea5ea563f5edc30071b4c508d6775f7cb5a557ca710135139c0646e5"}
Oct 31 03:28:02.003354 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:02.003323    2580 generic.go:356] "Generic (PLEG): container finished" podID="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" containerID="ddf56ba6766402bbff126a852c27b70d3df7421db2b5300a3f3918060b1b6688" exitCode=0
Oct 31 03:28:02.003637 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:02.003359    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtmjj" event={"ID":"9193a1f4-f4aa-4364-ae16-5520e88d8cc3","Type":"ContainerDied","Data":"ddf56ba6766402bbff126a852c27b70d3df7421db2b5300a3f3918060b1b6688"}
Oct 31 03:28:03.009319 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:03.009280    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xtmjj" event={"ID":"9193a1f4-f4aa-4364-ae16-5520e88d8cc3","Type":"ContainerStarted","Data":"bf0838093a232b0f010d9910bae803935c3133fb29630d98344ec7b484fa0576"}
Oct 31 03:28:03.815050 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:03.814871    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:03.815205 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:03.814935    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:28:03.815205 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:03.815125    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:28:03.815205 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:03.814972    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:03.815205 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:03.815190    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:28:03.815428 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:03.815301    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:28:04.015222 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:04.015184    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9" event={"ID":"ac8c7ff3-c156-4201-a297-ea6d1fd48e55","Type":"ContainerStarted","Data":"87f7fece974d166e815ca960431e2b6cebf4fd3de70cae7d1f9952221bb8d2eb"}
Oct 31 03:28:04.186327 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:04.186229    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xtmjj" podStartSLOduration=9.50101547 podStartE2EDuration="23.186215402s" totalImagesPullingTime="13.685199932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:03.247894159 +0000 UTC m=+22.865795580" watchObservedRunningTime="2026-10-31 03:28:04.186215402 +0000 UTC m=+23.804116821"
Oct 31 03:28:04.186738 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:04.186704    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9" podStartSLOduration=6.588956292 podStartE2EDuration="23.186694263s" totalImagesPullingTime="16.597737971s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 03:27:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:04.186115781 +0000 UTC m=+23.804017204" watchObservedRunningTime="2026-10-31 03:28:04.186694263 +0000 UTC m=+23.804595685"
Oct 31 03:28:05.017589 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:05.017555    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 03:28:05.017980 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:05.017963    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:05.018072 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:05.018058    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:05.049610 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:05.048808    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:05.049610 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:05.049079    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:05.815705 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:05.815681    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:05.815883 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:05.815757    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:05.815883 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:05.815844    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:28:05.815883 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:05.815848    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:28:05.816017 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:05.815929    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:28:05.816017 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:05.815975    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:28:06.019850 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:06.019825    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 03:28:06.106476 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:06.106409    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-phb2z"]
Oct 31 03:28:06.106606 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:06.106499    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:06.106606 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:06.106578    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:28:06.109474 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:06.109451    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n6jzp"]
Oct 31 03:28:06.109599 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:06.109548    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:28:06.109667 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:06.109646    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:28:06.112086 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:06.112065    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-65ph6"]
Oct 31 03:28:06.112159 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:06.112144    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:06.112249 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:06.112229    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:28:07.021703 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:07.021676    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 03:28:07.815730 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:07.815657    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:07.815730 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:07.815678    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:07.815919 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:07.815762    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:28:07.815919 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:07.815781    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:28:07.815919 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:07.815825    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:28:07.816025 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:07.815912    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:28:08.996034 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:08.996000    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:08.996527 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:08.996210    2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 03:28:09.007161 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:09.007118    2580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9" podUID="ac8c7ff3-c156-4201-a297-ea6d1fd48e55" containerName="ovnkube-controller" probeResult="failure" output=""
Oct 31 03:28:09.016261 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:09.016225    2580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9" podUID="ac8c7ff3-c156-4201-a297-ea6d1fd48e55" containerName="ovnkube-controller" probeResult="failure" output=""
Oct 31 03:28:09.814903 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:09.814866    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:09.815244 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:09.814915    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:09.815244 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:09.814972    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:28:09.815244 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:09.814985    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:28:09.815244 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:09.815048    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:28:09.815244 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:09.815093    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:28:11.814947 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:11.814754    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:11.815320 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:11.814832    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:11.815320 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:11.815000    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-65ph6" podUID="d78ab9e2-f553-441e-9f3d-6d8985085c89"
Oct 31 03:28:11.815320 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:11.815074    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-phb2z" podUID="609f4320-8b1b-459e-9e9b-9aeeed31dbe2"
Oct 31 03:28:11.815320 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:11.814842    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:28:11.815320 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:11.815148    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-n6jzp" podUID="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8"
Oct 31 03:28:12.188523 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.188452    2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeReady"
Oct 31 03:28:12.188652 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.188570    2580 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 31 03:28:12.272149 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.272118    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-8pxph"]
Oct 31 03:28:12.354967 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.354939    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-htdq6"]
Oct 31 03:28:12.355110 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.355058    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-8pxph"
Oct 31 03:28:12.367491 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.367470    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-kfthp"]
Oct 31 03:28:12.367604 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.367588    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-htdq6"
Oct 31 03:28:12.379141 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.379116    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"]
Oct 31 03:28:12.379356 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.379339    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-kfthp"
Oct 31 03:28:12.397385 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.397355    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"]
Oct 31 03:28:12.397670 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.397655    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:12.413851 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.413825    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-d6qdw"]
Oct 31 03:28:12.414059 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.414030    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:12.414284 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.414242    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 31 03:28:12.414370 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.414357    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 31 03:28:12.437638 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.437600    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-s42wk"]
Oct 31 03:28:12.437751 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.437728    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-d6qdw"
Oct 31 03:28:12.449809 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.449791    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 31 03:28:12.457124 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.457107    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-77685b76bf-cmqhp"]
Oct 31 03:28:12.457246 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.457231    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.464212 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.464191    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 31 03:28:12.474088 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.474072    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8bd6c7ddd-l84xn"]
Oct 31 03:28:12.474188 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.474174    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:12.488402 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.488380    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"]
Oct 31 03:28:12.488575 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.488559    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.502921 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.502903    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdrgg\" (UniqueName: \"kubernetes.io/projected/02dec91e-45a1-4609-9b09-72ba5259bd0e-kube-api-access-bdrgg\") pod \"kas-connection-checker-685895c7c9-kfthp\" (UID: \"02dec91e-45a1-4609-9b09-72ba5259bd0e\") " pod="kube-system/kas-connection-checker-685895c7c9-kfthp"
Oct 31 03:28:12.503007 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.502951    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:12.503007 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.502972    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j45vj\" (UniqueName: \"kubernetes.io/projected/bf18ddb4-723c-4954-8327-37f38b41eff1-kube-api-access-j45vj\") pod \"kas-connection-checker-685895c7c9-8pxph\" (UID: \"bf18ddb4-723c-4954-8327-37f38b41eff1\") " pod="kube-system/kas-connection-checker-685895c7c9-8pxph"
Oct 31 03:28:12.503074 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.503022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-nginx-conf\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:12.503074 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.503046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbg5x\" (UniqueName: \"kubernetes.io/projected/cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a-kube-api-access-pbg5x\") pod \"kas-connection-checker-685895c7c9-htdq6\" (UID: \"cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a\") " pod="kube-system/kas-connection-checker-685895c7c9-htdq6"
Oct 31 03:28:12.503130 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.503111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:12.503162 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.503130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxbxj\" (UniqueName: \"kubernetes.io/projected/aef16625-d836-42cb-bf49-176f80e73474-kube-api-access-gxbxj\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:12.503162 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.503153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aef16625-d836-42cb-bf49-176f80e73474-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:12.514765 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.514747    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"]
Oct 31 03:28:12.514879 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.514862    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:12.518167 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.518143    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 31 03:28:12.518167 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.518155    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 03:28:12.518167 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.518154    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:28:12.518384 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.518159    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 31 03:28:12.518384 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.518183    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 31 03:28:12.518384 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.518211    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 03:28:12.518384 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.518290    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 31 03:28:12.518622 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.518445    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 31 03:28:12.518787 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.518772    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 31 03:28:12.527366 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.527348    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"]
Oct 31 03:28:12.527478 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.527463    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:12.541214 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.541198    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"]
Oct 31 03:28:12.541313 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.541303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:12.558133 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.558117    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-5jgbv"]
Oct 31 03:28:12.558231 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.558195    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:12.566151 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.566127    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 31 03:28:12.566151 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.566138    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 31 03:28:12.566151 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.566145    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 31 03:28:12.566413 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.566388    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 31 03:28:12.566413 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.566385    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 31 03:28:12.566516 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.566417    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 31 03:28:12.566516 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.566424    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 31 03:28:12.566516 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.566448    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xg7xf\""
Oct 31 03:28:12.566516 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.566474    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 31 03:28:12.566516 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.566481    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 31 03:28:12.566516 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.566502    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 31 03:28:12.569848 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.566985    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 31 03:28:12.569848 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.567001    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 31 03:28:12.574180 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.574166    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5cb458f74f-7bjpx"]
Oct 31 03:28:12.574313 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.574300    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-5jgbv"
Oct 31 03:28:12.577230 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.577214    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 31 03:28:12.588462 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.588447    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zcpmp"]
Oct 31 03:28:12.588565 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.588552    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.604247 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604228    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:12.604358 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-nginx-conf\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:12.604358 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bb5340b5-c677-40d1-a448-498561c62a17-serving-cert\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:12.604470 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbg5x\" (UniqueName: \"kubernetes.io/projected/cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a-kube-api-access-pbg5x\") pod \"kas-connection-checker-685895c7c9-htdq6\" (UID: \"cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a\") " pod="kube-system/kas-connection-checker-685895c7c9-htdq6"
Oct 31 03:28:12.604470 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drh58\" (UniqueName: \"kubernetes.io/projected/0d9f6de8-463e-4779-9825-e4318b9273b4-kube-api-access-drh58\") pod \"volume-data-source-validator-58c9f5f46d-d6qdw\" (UID: \"0d9f6de8-463e-4779-9825-e4318b9273b4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-d6qdw"
Oct 31 03:28:12.604470 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604394    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:12.604470 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604411    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-certificates\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.604470 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-trusted-ca\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.604739 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604483    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df73b930-8122-434b-b2a0-cc3dd2c44d5c-serving-cert\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.604739 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-stats-auth\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:12.604739 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604560    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df73b930-8122-434b-b2a0-cc3dd2c44d5c-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.604739 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604586    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/df73b930-8122-434b-b2a0-cc3dd2c44d5c-snapshots\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.604739 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bb5340b5-c677-40d1-a448-498561c62a17-config\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:12.604739 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604625    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bb5340b5-c677-40d1-a448-498561c62a17-trusted-ca\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:12.604739 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604669    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:12.604739 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604738    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j45vj\" (UniqueName: \"kubernetes.io/projected/bf18ddb4-723c-4954-8327-37f38b41eff1-kube-api-access-j45vj\") pod \"kas-connection-checker-685895c7c9-8pxph\" (UID: \"bf18ddb4-723c-4954-8327-37f38b41eff1\") " pod="kube-system/kas-connection-checker-685895c7c9-8pxph"
Oct 31 03:28:12.605108 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.604751    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 03:28:12.605108 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-bound-sa-token\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.605108 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.604817    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert podName:f4b87fa6-eceb-4783-b2c5-4685e08ba9ef nodeName:}" failed. No retries permitted until 2026-10-31 03:28:13.104805935 +0000 UTC m=+32.722707335 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k2fcf" (UID: "f4b87fa6-eceb-4783-b2c5-4685e08ba9ef") : secret "networking-console-plugin-cert" not found
Oct 31 03:28:12.605108 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604876    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df73b930-8122-434b-b2a0-cc3dd2c44d5c-tmp\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.605108 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604920    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dfdt\" (UniqueName: \"kubernetes.io/projected/df73b930-8122-434b-b2a0-cc3dd2c44d5c-kube-api-access-7dfdt\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.605108 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604946    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l8rz\" (UniqueName: \"kubernetes.io/projected/26a79eda-be35-4aa8-b10b-cb2b1133a086-kube-api-access-2l8rz\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:12.605108 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604979    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r69c\" (UniqueName: \"kubernetes.io/projected/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-kube-api-access-4r69c\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:12.605108 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.604987    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-nginx-conf\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:12.605108 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.605016    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scqrb\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-kube-api-access-scqrb\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.605108 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.605048    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:12.605108 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.605068    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:12.605108 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.605095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-image-registry-private-configuration\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.605559 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.605151    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:12.605559 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.605202    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls podName:aef16625-d836-42cb-bf49-176f80e73474 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:13.105190802 +0000 UTC m=+32.723092202 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dwx2" (UID: "aef16625-d836-42cb-bf49-176f80e73474") : secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:12.605559 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.605252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxbxj\" (UniqueName: \"kubernetes.io/projected/aef16625-d836-42cb-bf49-176f80e73474-kube-api-access-gxbxj\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:12.605559 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.605312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-default-certificate\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:12.605559 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.605339    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df73b930-8122-434b-b2a0-cc3dd2c44d5c-service-ca-bundle\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.605559 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.605366    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:12.605559 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.605432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aef16625-d836-42cb-bf49-176f80e73474-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:12.605559 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.605497    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-installation-pull-secrets\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.605559 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.605528    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdrgg\" (UniqueName: \"kubernetes.io/projected/02dec91e-45a1-4609-9b09-72ba5259bd0e-kube-api-access-bdrgg\") pod \"kas-connection-checker-685895c7c9-kfthp\" (UID: \"02dec91e-45a1-4609-9b09-72ba5259bd0e\") " pod="kube-system/kas-connection-checker-685895c7c9-kfthp"
Oct 31 03:28:12.605559 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.605552    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.605949 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.605583    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05c25677-4b67-4a79-bbca-25bcc3ef43f1-ca-trust-extracted\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.605949 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.605641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-549vb\" (UniqueName: \"kubernetes.io/projected/bb5340b5-c677-40d1-a448-498561c62a17-kube-api-access-549vb\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:12.606012 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.605962    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aef16625-d836-42cb-bf49-176f80e73474-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:12.613168 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.613146    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-8pxph"]
Oct 31 03:28:12.613244 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.613175    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-htdq6"]
Oct 31 03:28:12.613244 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.613190    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-kfthp"]
Oct 31 03:28:12.613244 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.613207    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"]
Oct 31 03:28:12.613244 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.613229    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"]
Oct 31 03:28:12.613244 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.613238    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:12.613244 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.613247    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"]
Oct 31 03:28:12.613507 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.613262    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-s42wk"]
Oct 31 03:28:12.613507 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.613292    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8bd6c7ddd-l84xn"]
Oct 31 03:28:12.613507 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.613310    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cb458f74f-7bjpx"]
Oct 31 03:28:12.613507 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.613327    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-5jgbv"]
Oct 31 03:28:12.613507 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.613348    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pq8vh"]
Oct 31 03:28:12.615062 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.615047    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 31 03:28:12.615290 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.615257    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 31 03:28:12.615678 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.615579    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:28:12.615678 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.615597    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 31 03:28:12.615678 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.615597    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:28:12.615678 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.615619    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 31 03:28:12.615918 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.615630    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:28:12.615918 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.615643    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 31 03:28:12.615918 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.615661    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 31 03:28:12.615918 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.615636    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 31 03:28:12.615918 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.615838    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:28:12.615918 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.615842    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 31 03:28:12.616939 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.616924    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 31 03:28:12.617077 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.617063    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 31 03:28:12.624359 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.624339    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 03:28:12.628425 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.628409    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pq8vh"
Oct 31 03:28:12.628493 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.628411    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zcpmp"]
Oct 31 03:28:12.628493 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.628488    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77685b76bf-cmqhp"]
Oct 31 03:28:12.628603 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.628505    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-d6qdw"]
Oct 31 03:28:12.628603 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.628528    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"]
Oct 31 03:28:12.628603 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.628542    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"]
Oct 31 03:28:12.628603 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.628555    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"]
Oct 31 03:28:12.628603 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.628568    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pq8vh"]
Oct 31 03:28:12.650130 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.650110    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 31 03:28:12.650218 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.650110    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 31 03:28:12.665423 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.665404    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbg5x\" (UniqueName: \"kubernetes.io/projected/cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a-kube-api-access-pbg5x\") pod \"kas-connection-checker-685895c7c9-htdq6\" (UID: \"cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a\") " pod="kube-system/kas-connection-checker-685895c7c9-htdq6"
Oct 31 03:28:12.665512 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.665426    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j45vj\" (UniqueName: \"kubernetes.io/projected/bf18ddb4-723c-4954-8327-37f38b41eff1-kube-api-access-j45vj\") pod \"kas-connection-checker-685895c7c9-8pxph\" (UID: \"bf18ddb4-723c-4954-8327-37f38b41eff1\") " pod="kube-system/kas-connection-checker-685895c7c9-8pxph"
Oct 31 03:28:12.665512 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.665440    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdrgg\" (UniqueName: \"kubernetes.io/projected/02dec91e-45a1-4609-9b09-72ba5259bd0e-kube-api-access-bdrgg\") pod \"kas-connection-checker-685895c7c9-kfthp\" (UID: \"02dec91e-45a1-4609-9b09-72ba5259bd0e\") " pod="kube-system/kas-connection-checker-685895c7c9-kfthp"
Oct 31 03:28:12.665598 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.665584    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxbxj\" (UniqueName: \"kubernetes.io/projected/aef16625-d836-42cb-bf49-176f80e73474-kube-api-access-gxbxj\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:12.687466 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.687451    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-8pxph"
Oct 31 03:28:12.702586 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.702571    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-htdq6"
Oct 31 03:28:12.706849 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.706832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a52638f4-7a5b-45de-9054-d173d66c7521-config-volume\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:12.706913 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.706859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-549vb\" (UniqueName: \"kubernetes.io/projected/bb5340b5-c677-40d1-a448-498561c62a17-kube-api-access-549vb\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:12.706913 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.706878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.706913 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.706907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:12.707026 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.706942    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-bound-sa-token\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.707075 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707057    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-certificates\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.707118 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.707087    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 03:28:12.707164 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.707130    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:13.20711719 +0000 UTC m=+32.825018591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : secret "router-metrics-certs-default" not found
Oct 31 03:28:12.707164 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707144    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bb5340b5-c677-40d1-a448-498561c62a17-serving-cert\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:12.707251 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707169    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drh58\" (UniqueName: \"kubernetes.io/projected/0d9f6de8-463e-4779-9825-e4318b9273b4-kube-api-access-drh58\") pod \"volume-data-source-validator-58c9f5f46d-d6qdw\" (UID: \"0d9f6de8-463e-4779-9825-e4318b9273b4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-d6qdw"
Oct 31 03:28:12.707251 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:12.707251 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-certificates\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.707251 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707220    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sklx7\" (UniqueName: \"kubernetes.io/projected/8d7cafa8-5ce8-4991-8598-5e03ebfb0b75-kube-api-access-sklx7\") pod \"network-check-source-6f78d59f99-5jgbv\" (UID: \"8d7cafa8-5ce8-4991-8598-5e03ebfb0b75\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-5jgbv"
Oct 31 03:28:12.707251 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707239    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cc09f0c-627f-4445-b172-9556cab2bbdd-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg\" (UID: \"7cc09f0c-627f-4445-b172-9556cab2bbdd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:12.707498 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-trusted-ca\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.707498 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707300    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a52638f4-7a5b-45de-9054-d173d66c7521-tmp-dir\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:12.707498 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707375    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df73b930-8122-434b-b2a0-cc3dd2c44d5c-serving-cert\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.707498 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707417    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7cc09f0c-627f-4445-b172-9556cab2bbdd-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg\" (UID: \"7cc09f0c-627f-4445-b172-9556cab2bbdd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:12.707498 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707443    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0bf0c577-0537-46fd-950a-ce5f565a51da-config\") pod \"service-ca-operator-544d749c8c-nhj2n\" (UID: \"0bf0c577-0537-46fd-950a-ce5f565a51da\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:12.707498 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-image-registry-private-configuration\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.707785 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707558    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-stats-auth\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:12.707785 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707598    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df73b930-8122-434b-b2a0-cc3dd2c44d5c-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.707959 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707924    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-certificates\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.707959 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.707931    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:12.708113 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708092    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-trusted-ca\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.708246 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/df73b930-8122-434b-b2a0-cc3dd2c44d5c-snapshots\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.708459 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708289    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bb5340b5-c677-40d1-a448-498561c62a17-config\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:12.708459 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bb5340b5-c677-40d1-a448-498561c62a17-trusted-ca\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:12.708459 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5js2v\" (UniqueName: \"kubernetes.io/projected/7cc09f0c-627f-4445-b172-9556cab2bbdd-kube-api-access-5js2v\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg\" (UID: \"7cc09f0c-627f-4445-b172-9556cab2bbdd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:12.708459 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708375    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-trusted-ca\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.708459 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708392    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0bf0c577-0537-46fd-950a-ce5f565a51da-serving-cert\") pod \"service-ca-operator-544d749c8c-nhj2n\" (UID: \"0bf0c577-0537-46fd-950a-ce5f565a51da\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:12.708459 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-bound-sa-token\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.708459 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df73b930-8122-434b-b2a0-cc3dd2c44d5c-tmp\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.708459 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708504    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/260390fb-c26d-4989-9e8a-a104c398e1c6-ca-trust-extracted\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708532    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dfdt\" (UniqueName: \"kubernetes.io/projected/df73b930-8122-434b-b2a0-cc3dd2c44d5c-kube-api-access-7dfdt\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708554    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l8rz\" (UniqueName: \"kubernetes.io/projected/26a79eda-be35-4aa8-b10b-cb2b1133a086-kube-api-access-2l8rz\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708597    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r69c\" (UniqueName: \"kubernetes.io/projected/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-kube-api-access-4r69c\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708634    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a-cert\") pod \"ingress-canary-pq8vh\" (UID: \"fc6215a3-f0b6-4f89-9a0d-30589bb43c5a\") " pod="openshift-ingress-canary/ingress-canary-pq8vh"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scqrb\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-kube-api-access-scqrb\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708687    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708728    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-image-registry-private-configuration\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6q9r\" (UniqueName: \"kubernetes.io/projected/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a-kube-api-access-c6q9r\") pod \"ingress-canary-pq8vh\" (UID: \"fc6215a3-f0b6-4f89-9a0d-30589bb43c5a\") " pod="openshift-ingress-canary/ingress-canary-pq8vh"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-installation-pull-secrets\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708814    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-default-certificate\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708838    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df73b930-8122-434b-b2a0-cc3dd2c44d5c-service-ca-bundle\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708861    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/df73b930-8122-434b-b2a0-cc3dd2c44d5c-snapshots\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708866    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcq98\" (UniqueName: \"kubernetes.io/projected/a52638f4-7a5b-45de-9054-d173d66c7521-kube-api-access-gcq98\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:12.708948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708931    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfc5g\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-kube-api-access-pfc5g\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.709795 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.708968    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:13.208953372 +0000 UTC m=+32.826854792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : configmap references non-existent config key: service-ca.crt
Oct 31 03:28:12.709795 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.708986    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-installation-pull-secrets\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.709795 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.709014    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.709795 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.709039    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05c25677-4b67-4a79-bbca-25bcc3ef43f1-ca-trust-extracted\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.709795 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.709045    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df73b930-8122-434b-b2a0-cc3dd2c44d5c-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.709795 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.709067    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kk5j\" (UniqueName: \"kubernetes.io/projected/0bf0c577-0537-46fd-950a-ce5f565a51da-kube-api-access-6kk5j\") pod \"service-ca-operator-544d749c8c-nhj2n\" (UID: \"0bf0c577-0537-46fd-950a-ce5f565a51da\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:12.709795 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.709196    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 03:28:12.709795 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.709238    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls podName:cdbd24b7-07ab-44ba-a564-0b84466b7bfa nodeName:}" failed. No retries permitted until 2026-10-31 03:28:13.209224956 +0000 UTC m=+32.827126371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-cwnbr" (UID: "cdbd24b7-07ab-44ba-a564-0b84466b7bfa") : secret "samples-operator-tls" not found
Oct 31 03:28:12.709795 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.709336    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:12.709795 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.709349    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bd6c7ddd-l84xn: secret "image-registry-tls" not found
Oct 31 03:28:12.709795 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.709391    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls podName:05c25677-4b67-4a79-bbca-25bcc3ef43f1 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:13.209378471 +0000 UTC m=+32.827279886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls") pod "image-registry-8bd6c7ddd-l84xn" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1") : secret "image-registry-tls" not found
Oct 31 03:28:12.709795 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.709577    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bb5340b5-c677-40d1-a448-498561c62a17-config\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:12.709795 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.709717    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05c25677-4b67-4a79-bbca-25bcc3ef43f1-ca-trust-extracted\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.710443 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.709838    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bb5340b5-c677-40d1-a448-498561c62a17-serving-cert\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:12.710443 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.710167    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df73b930-8122-434b-b2a0-cc3dd2c44d5c-serving-cert\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.710443 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.710322    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df73b930-8122-434b-b2a0-cc3dd2c44d5c-service-ca-bundle\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.710443 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.710403    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df73b930-8122-434b-b2a0-cc3dd2c44d5c-tmp\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.710630 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.710540    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bb5340b5-c677-40d1-a448-498561c62a17-trusted-ca\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:12.711166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.711112    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-stats-auth\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:12.712120 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.712101    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-image-registry-private-configuration\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.712357 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.712341    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-installation-pull-secrets\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.712744 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.712731    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-default-certificate\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:12.715074 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.715054    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 03:28:12.715350 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.715331    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 03:28:12.715350 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.715335    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 03:28:12.715350 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.715355    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 03:28:12.718579 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.718561    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-kfthp"
Oct 31 03:28:12.751482 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.751459    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 03:28:12.751589 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.751467    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 03:28:12.751589 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.751493    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fs258\""
Oct 31 03:28:12.751669 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.751586    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-dvnpq\""
Oct 31 03:28:12.767381 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.767352    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drh58\" (UniqueName: \"kubernetes.io/projected/0d9f6de8-463e-4779-9825-e4318b9273b4-kube-api-access-drh58\") pod \"volume-data-source-validator-58c9f5f46d-d6qdw\" (UID: \"0d9f6de8-463e-4779-9825-e4318b9273b4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-d6qdw"
Oct 31 03:28:12.767756 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.767737    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-549vb\" (UniqueName: \"kubernetes.io/projected/bb5340b5-c677-40d1-a448-498561c62a17-kube-api-access-549vb\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:12.768915 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.768833    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dfdt\" (UniqueName: \"kubernetes.io/projected/df73b930-8122-434b-b2a0-cc3dd2c44d5c-kube-api-access-7dfdt\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.769499 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.769477    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scqrb\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-kube-api-access-scqrb\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.770040 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.770019    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r69c\" (UniqueName: \"kubernetes.io/projected/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-kube-api-access-4r69c\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:12.770401 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.770369    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-bound-sa-token\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:12.771593 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.771570    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l8rz\" (UniqueName: \"kubernetes.io/projected/26a79eda-be35-4aa8-b10b-cb2b1133a086-kube-api-access-2l8rz\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:12.774456 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.774434    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-d6qdw"
Oct 31 03:28:12.788627 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.788608    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:12.810534 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.809605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/260390fb-c26d-4989-9e8a-a104c398e1c6-ca-trust-extracted\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.810534 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.809652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:12.810534 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.809688    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a-cert\") pod \"ingress-canary-pq8vh\" (UID: \"fc6215a3-f0b6-4f89-9a0d-30589bb43c5a\") " pod="openshift-ingress-canary/ingress-canary-pq8vh"
Oct 31 03:28:12.810534 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.809742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6q9r\" (UniqueName: \"kubernetes.io/projected/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a-kube-api-access-c6q9r\") pod \"ingress-canary-pq8vh\" (UID: \"fc6215a3-f0b6-4f89-9a0d-30589bb43c5a\") " pod="openshift-ingress-canary/ingress-canary-pq8vh"
Oct 31 03:28:12.810534 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.809764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-installation-pull-secrets\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.810534 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.809810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcq98\" (UniqueName: \"kubernetes.io/projected/a52638f4-7a5b-45de-9054-d173d66c7521-kube-api-access-gcq98\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:12.810534 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.809836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfc5g\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-kube-api-access-pfc5g\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.810534 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.809886    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kk5j\" (UniqueName: \"kubernetes.io/projected/0bf0c577-0537-46fd-950a-ce5f565a51da-kube-api-access-6kk5j\") pod \"service-ca-operator-544d749c8c-nhj2n\" (UID: \"0bf0c577-0537-46fd-950a-ce5f565a51da\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:12.810534 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.810401    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 03:28:12.810534 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.810471    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls podName:a52638f4-7a5b-45de-9054-d173d66c7521 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:13.310449277 +0000 UTC m=+32.928350691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls") pod "dns-default-zcpmp" (UID: "a52638f4-7a5b-45de-9054-d173d66c7521") : secret "dns-default-metrics-tls" not found
Oct 31 03:28:12.811051 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.811019    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/260390fb-c26d-4989-9e8a-a104c398e1c6-ca-trust-extracted\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.811592 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.811571    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a52638f4-7a5b-45de-9054-d173d66c7521-config-volume\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:12.811661 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.811614    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.811661 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.811652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-bound-sa-token\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.811938 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.811696    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-certificates\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.811938 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.811745    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:12.811938 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.811764    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb458f74f-7bjpx: secret "image-registry-tls" not found
Oct 31 03:28:12.811938 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:12.811814    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls podName:260390fb-c26d-4989-9e8a-a104c398e1c6 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:13.311796376 +0000 UTC m=+32.929697789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls") pod "image-registry-5cb458f74f-7bjpx" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6") : secret "image-registry-tls" not found
Oct 31 03:28:12.811938 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.811757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sklx7\" (UniqueName: \"kubernetes.io/projected/8d7cafa8-5ce8-4991-8598-5e03ebfb0b75-kube-api-access-sklx7\") pod \"network-check-source-6f78d59f99-5jgbv\" (UID: \"8d7cafa8-5ce8-4991-8598-5e03ebfb0b75\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-5jgbv"
Oct 31 03:28:12.811938 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.811857    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cc09f0c-627f-4445-b172-9556cab2bbdd-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg\" (UID: \"7cc09f0c-627f-4445-b172-9556cab2bbdd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:12.811938 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.811889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a52638f4-7a5b-45de-9054-d173d66c7521-tmp-dir\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:12.811938 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.811927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7cc09f0c-627f-4445-b172-9556cab2bbdd-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg\" (UID: \"7cc09f0c-627f-4445-b172-9556cab2bbdd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:12.812361 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.811952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0bf0c577-0537-46fd-950a-ce5f565a51da-config\") pod \"service-ca-operator-544d749c8c-nhj2n\" (UID: \"0bf0c577-0537-46fd-950a-ce5f565a51da\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:12.812361 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.812030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-image-registry-private-configuration\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.812361 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.812058    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-trusted-ca\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.812361 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.812116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5js2v\" (UniqueName: \"kubernetes.io/projected/7cc09f0c-627f-4445-b172-9556cab2bbdd-kube-api-access-5js2v\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg\" (UID: \"7cc09f0c-627f-4445-b172-9556cab2bbdd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:12.812361 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.812157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0bf0c577-0537-46fd-950a-ce5f565a51da-serving-cert\") pod \"service-ca-operator-544d749c8c-nhj2n\" (UID: \"0bf0c577-0537-46fd-950a-ce5f565a51da\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:12.812593 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.812392    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a52638f4-7a5b-45de-9054-d173d66c7521-config-volume\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:12.812662 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.812641    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-certificates\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.813127 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.813102    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0bf0c577-0537-46fd-950a-ce5f565a51da-config\") pod \"service-ca-operator-544d749c8c-nhj2n\" (UID: \"0bf0c577-0537-46fd-950a-ce5f565a51da\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:12.813212 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.813157    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a52638f4-7a5b-45de-9054-d173d66c7521-tmp-dir\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:12.814189 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.813785    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7cc09f0c-627f-4445-b172-9556cab2bbdd-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg\" (UID: \"7cc09f0c-627f-4445-b172-9556cab2bbdd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:12.814540 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.814477    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-trusted-ca\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.820401 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.819804    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a-cert\") pod \"ingress-canary-pq8vh\" (UID: \"fc6215a3-f0b6-4f89-9a0d-30589bb43c5a\") " pod="openshift-ingress-canary/ingress-canary-pq8vh"
Oct 31 03:28:12.820401 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.819924    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-image-registry-private-configuration\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.820401 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.820077    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0bf0c577-0537-46fd-950a-ce5f565a51da-serving-cert\") pod \"service-ca-operator-544d749c8c-nhj2n\" (UID: \"0bf0c577-0537-46fd-950a-ce5f565a51da\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:12.821535 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.821489    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cc09f0c-627f-4445-b172-9556cab2bbdd-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg\" (UID: \"7cc09f0c-627f-4445-b172-9556cab2bbdd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:12.824703 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.824659    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-installation-pull-secrets\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.877844 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.876584    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-bound-sa-token\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.878713 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.878575    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5js2v\" (UniqueName: \"kubernetes.io/projected/7cc09f0c-627f-4445-b172-9556cab2bbdd-kube-api-access-5js2v\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg\" (UID: \"7cc09f0c-627f-4445-b172-9556cab2bbdd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:12.878713 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.878600    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kk5j\" (UniqueName: \"kubernetes.io/projected/0bf0c577-0537-46fd-950a-ce5f565a51da-kube-api-access-6kk5j\") pod \"service-ca-operator-544d749c8c-nhj2n\" (UID: \"0bf0c577-0537-46fd-950a-ce5f565a51da\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:12.878713 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.878579    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcq98\" (UniqueName: \"kubernetes.io/projected/a52638f4-7a5b-45de-9054-d173d66c7521-kube-api-access-gcq98\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:12.879138 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.879042    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfc5g\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-kube-api-access-pfc5g\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:12.884576 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.880201    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sklx7\" (UniqueName: \"kubernetes.io/projected/8d7cafa8-5ce8-4991-8598-5e03ebfb0b75-kube-api-access-sklx7\") pod \"network-check-source-6f78d59f99-5jgbv\" (UID: \"8d7cafa8-5ce8-4991-8598-5e03ebfb0b75\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-5jgbv"
Oct 31 03:28:12.884733 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.884570    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:12.885893 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.885497    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6q9r\" (UniqueName: \"kubernetes.io/projected/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a-kube-api-access-c6q9r\") pod \"ingress-canary-pq8vh\" (UID: \"fc6215a3-f0b6-4f89-9a0d-30589bb43c5a\") " pod="openshift-ingress-canary/ingress-canary-pq8vh"
Oct 31 03:28:12.900307 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.899469    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-5jgbv"
Oct 31 03:28:12.923331 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.923309    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:12.972605 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:12.972576    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pq8vh"
Oct 31 03:28:13.115302 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.115246    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:13.115458 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.115356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:13.115458 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.115415    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 03:28:13.115519 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.115486    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert podName:f4b87fa6-eceb-4783-b2c5-4685e08ba9ef nodeName:}" failed. No retries permitted until 2026-10-31 03:28:14.115467085 +0000 UTC m=+33.733368502 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k2fcf" (UID: "f4b87fa6-eceb-4783-b2c5-4685e08ba9ef") : secret "networking-console-plugin-cert" not found
Oct 31 03:28:13.115519 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.115506    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:13.115578 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.115573    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls podName:aef16625-d836-42cb-bf49-176f80e73474 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:14.115556174 +0000 UTC m=+33.733457576 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dwx2" (UID: "aef16625-d836-42cb-bf49-176f80e73474") : secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:13.133356 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.133238    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-htdq6"]
Oct 31 03:28:13.136764 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.136716    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-kfthp"]
Oct 31 03:28:13.137537 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.137509    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-8pxph"]
Oct 31 03:28:13.138564 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:28:13.138541    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf18ddb4_723c_4954_8327_37f38b41eff1.slice/crio-52e93485e8131c2278d113ade1bc0d730f241150bc93a844897c00b87de722b8 WatchSource:0}: Error finding container 52e93485e8131c2278d113ade1bc0d730f241150bc93a844897c00b87de722b8: Status 404 returned error can't find the container with id 52e93485e8131c2278d113ade1bc0d730f241150bc93a844897c00b87de722b8
Oct 31 03:28:13.139252 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:28:13.139233    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02dec91e_45a1_4609_9b09_72ba5259bd0e.slice/crio-87f7f97822d72850958495720216553b16fb5554dba8399e677ebbede15c3681 WatchSource:0}: Error finding container 87f7f97822d72850958495720216553b16fb5554dba8399e677ebbede15c3681: Status 404 returned error can't find the container with id 87f7f97822d72850958495720216553b16fb5554dba8399e677ebbede15c3681
Oct 31 03:28:13.170846 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.170632    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:13.216070 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.216038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:13.216198 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.216112    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:13.216198 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.216159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:13.216347 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.216219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:13.216499 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.216481    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 03:28:13.216574 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.216545    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 03:28:13.216628 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.216583    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:14.216562136 +0000 UTC m=+33.834463555 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : secret "router-metrics-certs-default" not found
Oct 31 03:28:13.216628 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.216610    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls podName:cdbd24b7-07ab-44ba-a564-0b84466b7bfa nodeName:}" failed. No retries permitted until 2026-10-31 03:28:14.216590502 +0000 UTC m=+33.834491904 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-cwnbr" (UID: "cdbd24b7-07ab-44ba-a564-0b84466b7bfa") : secret "samples-operator-tls" not found
Oct 31 03:28:13.216741 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.216637    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-s42wk"]
Oct 31 03:28:13.216741 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.216666    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:14.216655697 +0000 UTC m=+33.834557096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : configmap references non-existent config key: service-ca.crt
Oct 31 03:28:13.216741 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.216735    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:13.216891 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.216747    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bd6c7ddd-l84xn: secret "image-registry-tls" not found
Oct 31 03:28:13.216891 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.216777    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls podName:05c25677-4b67-4a79-bbca-25bcc3ef43f1 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:14.216767054 +0000 UTC m=+33.834668457 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls") pod "image-registry-8bd6c7ddd-l84xn" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1") : secret "image-registry-tls" not found
Oct 31 03:28:13.222732 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:28:13.222707    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf73b930_8122_434b_b2a0_cc3dd2c44d5c.slice/crio-bc146c9109a27614c348b38314d89af18695612cd24083d4005a28168fe9f068 WatchSource:0}: Error finding container bc146c9109a27614c348b38314d89af18695612cd24083d4005a28168fe9f068: Status 404 returned error can't find the container with id bc146c9109a27614c348b38314d89af18695612cd24083d4005a28168fe9f068
Oct 31 03:28:13.289886 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.289864    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"]
Oct 31 03:28:13.291922 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:28:13.291899    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5340b5_c677_40d1_a448_498561c62a17.slice/crio-bd6e84050794860cc4853cd9383b8adbb6463bacc36a757dcf79c40ce29bf13d WatchSource:0}: Error finding container bd6e84050794860cc4853cd9383b8adbb6463bacc36a757dcf79c40ce29bf13d: Status 404 returned error can't find the container with id bd6e84050794860cc4853cd9383b8adbb6463bacc36a757dcf79c40ce29bf13d
Oct 31 03:28:13.297758 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.297736    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-5jgbv"]
Oct 31 03:28:13.300087 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:28:13.300068    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d7cafa8_5ce8_4991_8598_5e03ebfb0b75.slice/crio-9bf724622b185f83fa250bf59ebd5ea37bec456e6b1d6336cc4bb00f11278944 WatchSource:0}: Error finding container 9bf724622b185f83fa250bf59ebd5ea37bec456e6b1d6336cc4bb00f11278944: Status 404 returned error can't find the container with id 9bf724622b185f83fa250bf59ebd5ea37bec456e6b1d6336cc4bb00f11278944
Oct 31 03:28:13.300727 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.300709    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-d6qdw"]
Oct 31 03:28:13.302355 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:28:13.302340    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d9f6de8_463e_4779_9825_e4318b9273b4.slice/crio-a26d728ea796615b99095298698a22f527fa1d67c9d2ca50ca1d973c156fde50 WatchSource:0}: Error finding container a26d728ea796615b99095298698a22f527fa1d67c9d2ca50ca1d973c156fde50: Status 404 returned error can't find the container with id a26d728ea796615b99095298698a22f527fa1d67c9d2ca50ca1d973c156fde50
Oct 31 03:28:13.317084 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.317065    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:13.317209 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.317192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:13.317681 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.317199    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 03:28:13.317681 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.317255    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:13.317681 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.317360    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls podName:a52638f4-7a5b-45de-9054-d173d66c7521 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:14.317337176 +0000 UTC m=+33.935238576 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls") pod "dns-default-zcpmp" (UID: "a52638f4-7a5b-45de-9054-d173d66c7521") : secret "dns-default-metrics-tls" not found
Oct 31 03:28:13.317681 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.317365    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb458f74f-7bjpx: secret "image-registry-tls" not found
Oct 31 03:28:13.317681 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.317401    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls podName:260390fb-c26d-4989-9e8a-a104c398e1c6 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:14.317389838 +0000 UTC m=+33.935291237 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls") pod "image-registry-5cb458f74f-7bjpx" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6") : secret "image-registry-tls" not found
Oct 31 03:28:13.320122 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.320104    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"]
Oct 31 03:28:13.321941 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:28:13.321917    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bf0c577_0537_46fd_950a_ce5f565a51da.slice/crio-a5c7d8bb069658a5c863248fbc65be1598745fe8dc4f185203058a3f55581490 WatchSource:0}: Error finding container a5c7d8bb069658a5c863248fbc65be1598745fe8dc4f185203058a3f55581490: Status 404 returned error can't find the container with id a5c7d8bb069658a5c863248fbc65be1598745fe8dc4f185203058a3f55581490
Oct 31 03:28:13.356659 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.356637    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pq8vh"]
Oct 31 03:28:13.358499 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:28:13.358472    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc6215a3_f0b6_4f89_9a0d_30589bb43c5a.slice/crio-bc01fe78c59ccb5472e9521b831780a720cf37ce4f1fdb34f65cdb0663490878 WatchSource:0}: Error finding container bc01fe78c59ccb5472e9521b831780a720cf37ce4f1fdb34f65cdb0663490878: Status 404 returned error can't find the container with id bc01fe78c59ccb5472e9521b831780a720cf37ce4f1fdb34f65cdb0663490878
Oct 31 03:28:13.570475 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.570453    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"]
Oct 31 03:28:13.572297 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:28:13.572254    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc09f0c_627f_4445_b172_9556cab2bbdd.slice/crio-021c9211dacc25c411efcc28b88172db25c870666c45876af00c3e0b3afcd9d3 WatchSource:0}: Error finding container 021c9211dacc25c411efcc28b88172db25c870666c45876af00c3e0b3afcd9d3: Status 404 returned error can't find the container with id 021c9211dacc25c411efcc28b88172db25c870666c45876af00c3e0b3afcd9d3
Oct 31 03:28:13.815472 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.815431    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:13.815472 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.815475    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:28:13.815706 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.815447    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:13.821910 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.821849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:13.822287 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.822007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:13.866055 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.866019    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 03:28:13.866189 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.866110    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vvwkt\""
Oct 31 03:28:13.866259 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.866233    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-lj989\""
Oct 31 03:28:13.866330 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.866259    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-8f2x4\""
Oct 31 03:28:13.866330 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.866319    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 03:28:13.872373 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.872357    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 03:28:13.872444 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:13.872411    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs podName:d78ab9e2-f553-441e-9f3d-6d8985085c89 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:45.872396861 +0000 UTC m=+65.490298262 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs") pod "network-metrics-daemon-65ph6" (UID: "d78ab9e2-f553-441e-9f3d-6d8985085c89") : secret "metrics-daemon-secret" not found
Oct 31 03:28:13.874628 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.874609    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:13.919151 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.919125    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:13.923385 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.923365    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56fwn\" (UniqueName: \"kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn\") pod \"network-check-target-n6jzp\" (UID: \"05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8\") " pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:28:13.925818 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:13.925793    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56fwn\" (UniqueName: \"kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn\") pod \"network-check-target-n6jzp\" (UID: \"05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8\") " pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:28:14.037363 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.037330    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg" event={"ID":"7cc09f0c-627f-4445-b172-9556cab2bbdd","Type":"ContainerStarted","Data":"021c9211dacc25c411efcc28b88172db25c870666c45876af00c3e0b3afcd9d3"}
Oct 31 03:28:14.038387 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.038367    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-5jgbv" event={"ID":"8d7cafa8-5ce8-4991-8598-5e03ebfb0b75","Type":"ContainerStarted","Data":"9bf724622b185f83fa250bf59ebd5ea37bec456e6b1d6336cc4bb00f11278944"}
Oct 31 03:28:14.039788 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.039759    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-htdq6" event={"ID":"cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a","Type":"ContainerStarted","Data":"d753f283d41b484151fff401bfe93b4d2e6c92715b7a78d0d82a9e8534f5ca43"}
Oct 31 03:28:14.039876 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.039797    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-htdq6" event={"ID":"cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a","Type":"ContainerStarted","Data":"9e05680ab6472f21635513f010c0f8f119a622a9aaefe68e7131da9079199fe0"}
Oct 31 03:28:14.040910 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.040891    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pq8vh" event={"ID":"fc6215a3-f0b6-4f89-9a0d-30589bb43c5a","Type":"ContainerStarted","Data":"bc01fe78c59ccb5472e9521b831780a720cf37ce4f1fdb34f65cdb0663490878"}
Oct 31 03:28:14.042239 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.042209    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-8pxph" event={"ID":"bf18ddb4-723c-4954-8327-37f38b41eff1","Type":"ContainerStarted","Data":"de63a0f6ec28c60e5771881ed1224ff367acca301b969f1aa8fd479c40341bfb"}
Oct 31 03:28:14.042342 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.042262    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-8pxph" event={"ID":"bf18ddb4-723c-4954-8327-37f38b41eff1","Type":"ContainerStarted","Data":"52e93485e8131c2278d113ade1bc0d730f241150bc93a844897c00b87de722b8"}
Oct 31 03:28:14.043351 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.043334    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n" event={"ID":"0bf0c577-0537-46fd-950a-ce5f565a51da","Type":"ContainerStarted","Data":"a5c7d8bb069658a5c863248fbc65be1598745fe8dc4f185203058a3f55581490"}
Oct 31 03:28:14.044457 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.044435    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm" event={"ID":"bb5340b5-c677-40d1-a448-498561c62a17","Type":"ContainerStarted","Data":"bd6e84050794860cc4853cd9383b8adbb6463bacc36a757dcf79c40ce29bf13d"}
Oct 31 03:28:14.045574 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.045553    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-s42wk" event={"ID":"df73b930-8122-434b-b2a0-cc3dd2c44d5c","Type":"ContainerStarted","Data":"bc146c9109a27614c348b38314d89af18695612cd24083d4005a28168fe9f068"}
Oct 31 03:28:14.047283 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.047246    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-kfthp" event={"ID":"02dec91e-45a1-4609-9b09-72ba5259bd0e","Type":"ContainerStarted","Data":"ce02459ed394e57691a20ae7b3f41b5e4daeb461be9bffb9cbdd55606a3cda67"}
Oct 31 03:28:14.047356 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.047298    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-kfthp" event={"ID":"02dec91e-45a1-4609-9b09-72ba5259bd0e","Type":"ContainerStarted","Data":"87f7f97822d72850958495720216553b16fb5554dba8399e677ebbede15c3681"}
Oct 31 03:28:14.050425 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.050405    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-d6qdw" event={"ID":"0d9f6de8-463e-4779-9825-e4318b9273b4","Type":"ContainerStarted","Data":"a26d728ea796615b99095298698a22f527fa1d67c9d2ca50ca1d973c156fde50"}
Oct 31 03:28:14.125037 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.124961    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:14.125175 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.125077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:14.125175 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.125139    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:14.125175 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.125157    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 03:28:14.125333 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.125203    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert podName:f4b87fa6-eceb-4783-b2c5-4685e08ba9ef nodeName:}" failed. No retries permitted until 2026-10-31 03:28:16.12518991 +0000 UTC m=+35.743091310 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k2fcf" (UID: "f4b87fa6-eceb-4783-b2c5-4685e08ba9ef") : secret "networking-console-plugin-cert" not found
Oct 31 03:28:14.125333 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.125216    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls podName:aef16625-d836-42cb-bf49-176f80e73474 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:16.125209389 +0000 UTC m=+35.743110788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dwx2" (UID: "aef16625-d836-42cb-bf49-176f80e73474") : secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:14.203214 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.203190    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:28:14.225677 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.225651    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:14.225807 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.225705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:14.225807 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.225755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:14.225929 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.225828    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:14.225929 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.225842    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bd6c7ddd-l84xn: secret "image-registry-tls" not found
Oct 31 03:28:14.225929 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.225889    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls podName:05c25677-4b67-4a79-bbca-25bcc3ef43f1 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:16.225874005 +0000 UTC m=+35.843775419 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls") pod "image-registry-8bd6c7ddd-l84xn" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1") : secret "image-registry-tls" not found
Oct 31 03:28:14.225929 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.225886    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 03:28:14.225929 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.225917    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls podName:cdbd24b7-07ab-44ba-a564-0b84466b7bfa nodeName:}" failed. No retries permitted until 2026-10-31 03:28:16.225909945 +0000 UTC m=+35.843811344 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-cwnbr" (UID: "cdbd24b7-07ab-44ba-a564-0b84466b7bfa") : secret "samples-operator-tls" not found
Oct 31 03:28:14.226166 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.225949    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:16.225932365 +0000 UTC m=+35.843833766 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : configmap references non-existent config key: service-ca.crt
Oct 31 03:28:14.226166 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.226040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:14.226240 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.226161    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 03:28:14.226240 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.226208    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:16.226194468 +0000 UTC m=+35.844095869 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : secret "router-metrics-certs-default" not found
Oct 31 03:28:14.298021 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.297990    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-phb2z"]
Oct 31 03:28:14.326677 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.326646    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:14.326789 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.326752    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:14.326840 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.326795    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 03:28:14.326871 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.326845    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls podName:a52638f4-7a5b-45de-9054-d173d66c7521 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:16.326831563 +0000 UTC m=+35.944732976 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls") pod "dns-default-zcpmp" (UID: "a52638f4-7a5b-45de-9054-d173d66c7521") : secret "dns-default-metrics-tls" not found
Oct 31 03:28:14.326871 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.326866    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:14.326950 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.326877    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb458f74f-7bjpx: secret "image-registry-tls" not found
Oct 31 03:28:14.326950 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:14.326928    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls podName:260390fb-c26d-4989-9e8a-a104c398e1c6 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:16.326917015 +0000 UTC m=+35.944818415 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls") pod "image-registry-5cb458f74f-7bjpx" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6") : secret "image-registry-tls" not found
Oct 31 03:28:14.410722 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.410627    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-htdq6" podStartSLOduration=399.410614044 podStartE2EDuration="6m39.410614044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:21:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:14.410068184 +0000 UTC m=+34.027969606" watchObservedRunningTime="2026-10-31 03:28:14.410614044 +0000 UTC m=+34.028515463"
Oct 31 03:28:14.591867 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.591776    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-kfthp" podStartSLOduration=399.591756758 podStartE2EDuration="6m39.591756758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:21:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:14.521149923 +0000 UTC m=+34.139051344" watchObservedRunningTime="2026-10-31 03:28:14.591756758 +0000 UTC m=+34.209658183"
Oct 31 03:28:14.593025 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.592998    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-n6jzp"]
Oct 31 03:28:14.630890 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:14.630831    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-8pxph" podStartSLOduration=399.630818353 podStartE2EDuration="6m39.630818353s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:21:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:14.630495272 +0000 UTC m=+34.248396693" watchObservedRunningTime="2026-10-31 03:28:14.630818353 +0000 UTC m=+34.248719797"
Oct 31 03:28:15.057950 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:15.057879    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-phb2z" event={"ID":"609f4320-8b1b-459e-9e9b-9aeeed31dbe2","Type":"ContainerStarted","Data":"642584615ebce25cc91736eb61b391b4458f0fc1c67a4cc8b1f1194a58f4406a"}
Oct 31 03:28:15.060515 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:15.060485    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n6jzp" event={"ID":"05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8","Type":"ContainerStarted","Data":"e49334cf60cfa11b1eebfe5d37557621069c99e7e5d28658af5ca201f7d8b9cb"}
Oct 31 03:28:16.147762 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:16.146919    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:16.147762 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:16.147003    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:16.147762 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.147168    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:16.147762 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.147232    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls podName:aef16625-d836-42cb-bf49-176f80e73474 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:20.147214395 +0000 UTC m=+39.765115798 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dwx2" (UID: "aef16625-d836-42cb-bf49-176f80e73474") : secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:16.147762 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.147667    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 03:28:16.147762 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.147722    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert podName:f4b87fa6-eceb-4783-b2c5-4685e08ba9ef nodeName:}" failed. No retries permitted until 2026-10-31 03:28:20.147705533 +0000 UTC m=+39.765606949 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k2fcf" (UID: "f4b87fa6-eceb-4783-b2c5-4685e08ba9ef") : secret "networking-console-plugin-cert" not found
Oct 31 03:28:16.248609 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:16.247842    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:16.248609 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:16.247980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:16.248609 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:16.248036    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:16.248609 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:16.248074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:16.248609 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.248211    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:16.248609 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.248226    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bd6c7ddd-l84xn: secret "image-registry-tls" not found
Oct 31 03:28:16.248609 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.248302    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls podName:05c25677-4b67-4a79-bbca-25bcc3ef43f1 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:20.248282422 +0000 UTC m=+39.866183851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls") pod "image-registry-8bd6c7ddd-l84xn" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1") : secret "image-registry-tls" not found
Oct 31 03:28:16.248609 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.248386    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 03:28:16.248609 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.248423    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:20.248409433 +0000 UTC m=+39.866310848 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : secret "router-metrics-certs-default" not found
Oct 31 03:28:16.248609 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.248475    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 03:28:16.248609 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.248504    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls podName:cdbd24b7-07ab-44ba-a564-0b84466b7bfa nodeName:}" failed. No retries permitted until 2026-10-31 03:28:20.24849465 +0000 UTC m=+39.866396065 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-cwnbr" (UID: "cdbd24b7-07ab-44ba-a564-0b84466b7bfa") : secret "samples-operator-tls" not found
Oct 31 03:28:16.248609 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.248574    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:20.248563782 +0000 UTC m=+39.866465191 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : configmap references non-existent config key: service-ca.crt
Oct 31 03:28:16.351324 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:16.350819    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:16.351324 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:16.350907    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:16.351324 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.351071    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:16.351324 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.351083    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb458f74f-7bjpx: secret "image-registry-tls" not found
Oct 31 03:28:16.351324 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.351128    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls podName:260390fb-c26d-4989-9e8a-a104c398e1c6 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:20.351115794 +0000 UTC m=+39.969017193 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls") pod "image-registry-5cb458f74f-7bjpx" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6") : secret "image-registry-tls" not found
Oct 31 03:28:16.351324 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.351180    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 03:28:16.351324 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:16.351200    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls podName:a52638f4-7a5b-45de-9054-d173d66c7521 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:20.351193893 +0000 UTC m=+39.969095292 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls") pod "dns-default-zcpmp" (UID: "a52638f4-7a5b-45de-9054-d173d66c7521") : secret "dns-default-metrics-tls" not found
Oct 31 03:28:17.075542 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:17.075460    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-d6qdw" event={"ID":"0d9f6de8-463e-4779-9825-e4318b9273b4","Type":"ContainerStarted","Data":"f37661efe4184bd83ba6a5e7c82949befd11c9699ffb5dda543212ea5e36a027"}
Oct 31 03:28:17.190080 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:17.189556    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-d6qdw" podStartSLOduration=372.707022488 podStartE2EDuration="6m16.189537495s" totalImagesPullingTime="3.482515007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:17.188669637 +0000 UTC m=+36.806571060" watchObservedRunningTime="2026-10-31 03:28:17.189537495 +0000 UTC m=+36.807438920"
Oct 31 03:28:19.086320 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:19.086221    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/0.log"
Oct 31 03:28:19.087283 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:19.087215    2580 scope.go:122] "RemoveContainer" containerID="448ee3b7f114ba6441318048f04b29e804f2e8c67d2a1cf5746049782186f354"
Oct 31 03:28:19.089803 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:19.089353    2580 generic.go:356] "Generic (PLEG): container finished" podID="bb5340b5-c677-40d1-a448-498561c62a17" containerID="448ee3b7f114ba6441318048f04b29e804f2e8c67d2a1cf5746049782186f354" exitCode=255
Oct 31 03:28:19.090710 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:19.090400    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm" event={"ID":"bb5340b5-c677-40d1-a448-498561c62a17","Type":"ContainerDied","Data":"448ee3b7f114ba6441318048f04b29e804f2e8c67d2a1cf5746049782186f354"}
Oct 31 03:28:19.097728 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:19.093683    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-s42wk" event={"ID":"df73b930-8122-434b-b2a0-cc3dd2c44d5c","Type":"ContainerStarted","Data":"1f77b08ba82cb7a1aec40762285db923b9970cf488b40e6e2976edd63df9e39d"}
Oct 31 03:28:20.100027 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.100000    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/1.log"
Oct 31 03:28:20.101641 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.100756    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/0.log"
Oct 31 03:28:20.101641 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.100805    2580 generic.go:356] "Generic (PLEG): container finished" podID="bb5340b5-c677-40d1-a448-498561c62a17" containerID="1b470c024d032c0301032eaa971e8b3d98823c8fa645a9638b1a6a876ccea7ea" exitCode=255
Oct 31 03:28:20.101641 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.100900    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm" event={"ID":"bb5340b5-c677-40d1-a448-498561c62a17","Type":"ContainerDied","Data":"1b470c024d032c0301032eaa971e8b3d98823c8fa645a9638b1a6a876ccea7ea"}
Oct 31 03:28:20.101641 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.100951    2580 scope.go:122] "RemoveContainer" containerID="448ee3b7f114ba6441318048f04b29e804f2e8c67d2a1cf5746049782186f354"
Oct 31 03:28:20.101641 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.101203    2580 scope.go:122] "RemoveContainer" containerID="1b470c024d032c0301032eaa971e8b3d98823c8fa645a9638b1a6a876ccea7ea"
Oct 31 03:28:20.101641 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.101418    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-wpxfm_openshift-console-operator(bb5340b5-c677-40d1-a448-498561c62a17)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm" podUID="bb5340b5-c677-40d1-a448-498561c62a17"
Oct 31 03:28:20.104615 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.104590    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg" event={"ID":"7cc09f0c-627f-4445-b172-9556cab2bbdd","Type":"ContainerStarted","Data":"9621d3494d28955f9d184c4f8ed64231ff1d8c03df7fb4370b1f047947704b16"}
Oct 31 03:28:20.196219 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.195534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:20.196219 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.195621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:20.196219 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.195735    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:20.196219 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.195784    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls podName:aef16625-d836-42cb-bf49-176f80e73474 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.195768223 +0000 UTC m=+47.813669627 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dwx2" (UID: "aef16625-d836-42cb-bf49-176f80e73474") : secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:20.196551 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.196529    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 03:28:20.196711 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.196587    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert podName:f4b87fa6-eceb-4783-b2c5-4685e08ba9ef nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.196569568 +0000 UTC m=+47.814470970 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k2fcf" (UID: "f4b87fa6-eceb-4783-b2c5-4685e08ba9ef") : secret "networking-console-plugin-cert" not found
Oct 31 03:28:20.216102 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.216059    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-s42wk" podStartSLOduration=373.725221679 podStartE2EDuration="6m19.216047475s" totalImagesPullingTime="5.490825796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:19.334175722 +0000 UTC m=+38.952077137" watchObservedRunningTime="2026-10-31 03:28:20.216047475 +0000 UTC m=+39.833948895"
Oct 31 03:28:20.296712 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.296680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:20.296884 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.296781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:20.296884 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.296826    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:20.296884 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.296859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:20.297041 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.297005    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:20.297041 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.297021    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bd6c7ddd-l84xn: secret "image-registry-tls" not found
Oct 31 03:28:20.297133 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.297079    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls podName:05c25677-4b67-4a79-bbca-25bcc3ef43f1 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.29706127 +0000 UTC m=+47.914962694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls") pod "image-registry-8bd6c7ddd-l84xn" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1") : secret "image-registry-tls" not found
Oct 31 03:28:20.297196 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.297155    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 03:28:20.297196 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.297191    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.297180005 +0000 UTC m=+47.915081406 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : secret "router-metrics-certs-default" not found
Oct 31 03:28:20.297363 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.297249    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 03:28:20.297363 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.297307    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls podName:cdbd24b7-07ab-44ba-a564-0b84466b7bfa nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.297294131 +0000 UTC m=+47.915195548 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-cwnbr" (UID: "cdbd24b7-07ab-44ba-a564-0b84466b7bfa") : secret "samples-operator-tls" not found
Oct 31 03:28:20.297490 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.297379    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.297370048 +0000 UTC m=+47.915271461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : configmap references non-existent config key: service-ca.crt
Oct 31 03:28:20.398051 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.397993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:20.398228 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.398128    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:20.398345 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.398323    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:20.398405 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.398348    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb458f74f-7bjpx: secret "image-registry-tls" not found
Oct 31 03:28:20.398405 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.398402    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls podName:260390fb-c26d-4989-9e8a-a104c398e1c6 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.398387565 +0000 UTC m=+48.016288998 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls") pod "image-registry-5cb458f74f-7bjpx" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6") : secret "image-registry-tls" not found
Oct 31 03:28:20.398538 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.398472    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 03:28:20.398538 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.398500    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls podName:a52638f4-7a5b-45de-9054-d173d66c7521 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.398489272 +0000 UTC m=+48.016390684 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls") pod "dns-default-zcpmp" (UID: "a52638f4-7a5b-45de-9054-d173d66c7521") : secret "dns-default-metrics-tls" not found
Oct 31 03:28:20.418712 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.418655    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg" podStartSLOduration=372.07103346 podStartE2EDuration="6m18.418634974s" totalImagesPullingTime="6.347601514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:20.327467956 +0000 UTC m=+39.945369381" watchObservedRunningTime="2026-10-31 03:28:20.418634974 +0000 UTC m=+40.036536397"
Oct 31 03:28:20.419390 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.419368    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tw5k4"]
Oct 31 03:28:20.448060 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.447979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:20.478156 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.478134    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tw5k4"]
Oct 31 03:28:20.498769 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.498725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5615d56d-7e1b-4f88-803c-b6a1c0435263-data-volume\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:20.498769 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.498768    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:20.498984 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.498796    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7j7n\" (UniqueName: \"kubernetes.io/projected/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-api-access-b7j7n\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:20.498984 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.498832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5615d56d-7e1b-4f88-803c-b6a1c0435263-crio-socket\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:20.498984 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.498884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:20.499353 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.499329    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 03:28:20.499566 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.499548    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jdz67\""
Oct 31 03:28:20.499653 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.499627    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 03:28:20.600293 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.599548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5615d56d-7e1b-4f88-803c-b6a1c0435263-data-volume\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:20.600293 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.599578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:20.600293 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.599595    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7j7n\" (UniqueName: \"kubernetes.io/projected/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-api-access-b7j7n\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:20.600293 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.599615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5615d56d-7e1b-4f88-803c-b6a1c0435263-crio-socket\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:20.600293 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.599650    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:20.600293 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.599850    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 03:28:20.600293 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.599895    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls podName:5615d56d-7e1b-4f88-803c-b6a1c0435263 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:21.099883146 +0000 UTC m=+40.717784545 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tw5k4" (UID: "5615d56d-7e1b-4f88-803c-b6a1c0435263") : secret "insights-runtime-extractor-tls" not found
Oct 31 03:28:20.600293 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.600162    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5615d56d-7e1b-4f88-803c-b6a1c0435263-data-volume\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:20.600293 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.600219    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 03:28:20.600293 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:20.600251    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm podName:5615d56d-7e1b-4f88-803c-b6a1c0435263 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:21.100241328 +0000 UTC m=+40.718142728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tw5k4" (UID: "5615d56d-7e1b-4f88-803c-b6a1c0435263") : configmap "kube-rbac-proxy" not found
Oct 31 03:28:20.600808 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.600779    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5615d56d-7e1b-4f88-803c-b6a1c0435263-crio-socket\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:20.662317 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:20.662286    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7j7n\" (UniqueName: \"kubernetes.io/projected/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-api-access-b7j7n\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:21.105046 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.105016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:21.105487 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.105107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:21.105487 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:21.105148    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 03:28:21.105487 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:21.105222    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm podName:5615d56d-7e1b-4f88-803c-b6a1c0435263 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:22.105202567 +0000 UTC m=+41.723103983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tw5k4" (UID: "5615d56d-7e1b-4f88-803c-b6a1c0435263") : configmap "kube-rbac-proxy" not found
Oct 31 03:28:21.105487 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:21.105262    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 03:28:21.105487 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:21.105336    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls podName:5615d56d-7e1b-4f88-803c-b6a1c0435263 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:22.105319258 +0000 UTC m=+41.723220658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tw5k4" (UID: "5615d56d-7e1b-4f88-803c-b6a1c0435263") : secret "insights-runtime-extractor-tls" not found
Oct 31 03:28:21.110587 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.110557    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pq8vh" event={"ID":"fc6215a3-f0b6-4f89-9a0d-30589bb43c5a","Type":"ContainerStarted","Data":"54d309607a5d0384d8625bf321cf04ab44792cc248cfa6289cd34826c03d4717"}
Oct 31 03:28:21.112969 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.112940    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n" event={"ID":"0bf0c577-0537-46fd-950a-ce5f565a51da","Type":"ContainerStarted","Data":"6cd70920bbea0d3c4edad13fa6154c908830964231e769b92b1e59d67b9ecb9f"}
Oct 31 03:28:21.115121 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.115098    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/1.log"
Oct 31 03:28:21.115742 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.115726    2580 scope.go:122] "RemoveContainer" containerID="1b470c024d032c0301032eaa971e8b3d98823c8fa645a9638b1a6a876ccea7ea"
Oct 31 03:28:21.115893 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:21.115867    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-wpxfm_openshift-console-operator(bb5340b5-c677-40d1-a448-498561c62a17)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm" podUID="bb5340b5-c677-40d1-a448-498561c62a17"
Oct 31 03:28:21.224447 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.224384    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pq8vh" podStartSLOduration=2.240794831 podStartE2EDuration="9.224366523s" totalImagesPullingTime="6.983571692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:28:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:21.223243641 +0000 UTC m=+40.841145089" watchObservedRunningTime="2026-10-31 03:28:21.224366523 +0000 UTC m=+40.842267947"
Oct 31 03:28:21.451375 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.451317    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n" podStartSLOduration=373.629071729 podStartE2EDuration="6m20.451304372s" totalImagesPullingTime="6.822232643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:21.33785585 +0000 UTC m=+40.955757288" watchObservedRunningTime="2026-10-31 03:28:21.451304372 +0000 UTC m=+41.069205795"
Oct 31 03:28:21.734769 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.734699    2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-mft5z"]
Oct 31 03:28:21.755681 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.755650    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mft5z"
Oct 31 03:28:21.790948 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.790920    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-mft5z"]
Oct 31 03:28:21.812401 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.812371    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7wqg\" (UniqueName: \"kubernetes.io/projected/945a916b-41c5-46f6-9c1a-9a8c93f7d6d2-kube-api-access-h7wqg\") pod \"migrator-66df5d496b-mft5z\" (UID: \"945a916b-41c5-46f6-9c1a-9a8c93f7d6d2\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mft5z"
Oct 31 03:28:21.895936 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.895911    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 31 03:28:21.895936 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.895931    2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 31 03:28:21.896171 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.896034    2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-q48mz\""
Oct 31 03:28:21.913703 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.913675    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7wqg\" (UniqueName: \"kubernetes.io/projected/945a916b-41c5-46f6-9c1a-9a8c93f7d6d2-kube-api-access-h7wqg\") pod \"migrator-66df5d496b-mft5z\" (UID: \"945a916b-41c5-46f6-9c1a-9a8c93f7d6d2\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mft5z"
Oct 31 03:28:21.972630 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:21.972597    2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7wqg\" (UniqueName: \"kubernetes.io/projected/945a916b-41c5-46f6-9c1a-9a8c93f7d6d2-kube-api-access-h7wqg\") pod \"migrator-66df5d496b-mft5z\" (UID: \"945a916b-41c5-46f6-9c1a-9a8c93f7d6d2\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mft5z"
Oct 31 03:28:22.082227 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:22.082190    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mft5z"
Oct 31 03:28:22.122292 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:22.120478    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:22.122292 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:22.120656    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 03:28:22.122292 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:22.120671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:22.122292 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:22.120727    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls podName:5615d56d-7e1b-4f88-803c-b6a1c0435263 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:24.120705194 +0000 UTC m=+43.738606613 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tw5k4" (UID: "5615d56d-7e1b-4f88-803c-b6a1c0435263") : secret "insights-runtime-extractor-tls" not found
Oct 31 03:28:22.122292 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:22.120739    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 03:28:22.122292 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:22.120792    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm podName:5615d56d-7e1b-4f88-803c-b6a1c0435263 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:24.120776294 +0000 UTC m=+43.738677714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tw5k4" (UID: "5615d56d-7e1b-4f88-803c-b6a1c0435263") : configmap "kube-rbac-proxy" not found
Oct 31 03:28:22.126813 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:22.126292    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-5jgbv" event={"ID":"8d7cafa8-5ce8-4991-8598-5e03ebfb0b75","Type":"ContainerStarted","Data":"16bd2707c984033510b07382d34c4d53d9915377ea65b6b299a26b0afad6bf41"}
Oct 31 03:28:22.128073 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:22.128048    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-n6jzp" event={"ID":"05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8","Type":"ContainerStarted","Data":"f9bde27b5af7e84714a1b9a36d4e5a11a29177a8b8d432ac9491d17f058ac016"}
Oct 31 03:28:22.237658 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:22.237613    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-5jgbv" podStartSLOduration=369.51203845 podStartE2EDuration="6m17.237600118s" totalImagesPullingTime="7.725561668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:22:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:22.236602409 +0000 UTC m=+41.854503832" watchObservedRunningTime="2026-10-31 03:28:22.237600118 +0000 UTC m=+41.855501544"
Oct 31 03:28:22.347200 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:22.347093    2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-n6jzp" podStartSLOduration=33.434043293 podStartE2EDuration="40.347078564s" totalImagesPullingTime="6.913035271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:22.345943866 +0000 UTC m=+41.963845289" watchObservedRunningTime="2026-10-31 03:28:22.347078564 +0000 UTC m=+41.964979986"
Oct 31 03:28:22.468990 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:22.468954    2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-mft5z"]
Oct 31 03:28:22.473176 ip-10-0-130-19 kubenswrapper[2580]: W1031 03:28:22.473142    2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod945a916b_41c5_46f6_9c1a_9a8c93f7d6d2.slice/crio-2a270b7bcead4afcea718fce44060043418f249c2f45a3345e912124014faabc WatchSource:0}: Error finding container 2a270b7bcead4afcea718fce44060043418f249c2f45a3345e912124014faabc: Status 404 returned error can't find the container with id 2a270b7bcead4afcea718fce44060043418f249c2f45a3345e912124014faabc
Oct 31 03:28:22.885204 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:22.885162    2580 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:22.885375 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:22.885221    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:22.885607 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:22.885591    2580 scope.go:122] "RemoveContainer" containerID="1b470c024d032c0301032eaa971e8b3d98823c8fa645a9638b1a6a876ccea7ea"
Oct 31 03:28:22.885803 ip-10-0-130-19 kubenswrapper[2580]: E1031 03:28:22.885783    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-wpxfm_openshift-console-operator(bb5340b5-c677-40d1-a448-498561c62a17)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm" podUID="bb5340b5-c677-40d1-a448-498561c62a17"
Oct 31 03:28:23.132943 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:23.132909    2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mft5z" event={"ID":"945a916b-41c5-46f6-9c1a-9a8c93f7d6d2","Type":"ContainerStarted","Data":"2a270b7bcead4afcea718fce44060043418f249c2f45a3345e912124014faabc"}
Oct 31 03:28:23.133317 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:23.133168    2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:28:23.331753 ip-10-0-130-19 kubenswrapper[2580]: I1031 03:28:23.331672    2580 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 03:28:23.331859 ip-10-0-130-19 systemd[1]: Stopping Kubernetes Kubelet...
Oct 31 03:28:23.348733 ip-10-0-130-19 systemd[1]: kubelet.service: Deactivated successfully.
Oct 31 03:28:23.348947 ip-10-0-130-19 systemd[1]: Stopped Kubernetes Kubelet.
Oct 31 03:28:23.348996 ip-10-0-130-19 systemd[1]: kubelet.service: Consumed 3.030s CPU time.
Oct 31 03:28:23.361964 ip-10-0-130-19 systemd[1]: Starting Kubernetes Kubelet...
Oct 31 03:28:23.442407 ip-10-0-130-19 kubenswrapper[4926]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 03:28:23.442407 ip-10-0-130-19 kubenswrapper[4926]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 31 03:28:23.442407 ip-10-0-130-19 kubenswrapper[4926]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 03:28:23.442407 ip-10-0-130-19 kubenswrapper[4926]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 31 03:28:23.444607 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444534    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:28:23.444607 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444607    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444611    4926 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444614    4926 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444618    4926 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444621    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444623    4926 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444626    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444628    4926 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444631    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444634    4926 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444636    4926 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444639    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444641    4926 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444644    4926 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444646    4926 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444649    4926 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444651    4926 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444654    4926 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444657    4926 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444666    4926 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:28:23.444677 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444668    4926 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444671    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444674    4926 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444677    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444680    4926 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444682    4926 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444686    4926 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444688    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444691    4926 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444693    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444696    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444698    4926 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444701    4926 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444703    4926 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444705    4926 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444708    4926 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444710    4926 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444713    4926 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444715    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444718    4926 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:28:23.445161 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444720    4926 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444723    4926 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444725    4926 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444727    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444730    4926 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444733    4926 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444735    4926 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444737    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444740    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444743    4926 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444746    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444749    4926 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444751    4926 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444754    4926 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444757    4926 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444760    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444762    4926 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444765    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444767    4926 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:28:23.445670 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444769    4926 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444772    4926 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444774    4926 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444777    4926 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444779    4926 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444781    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444784    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444786    4926 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444789    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444791    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444794    4926 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444797    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444800    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444803    4926 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444805    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444809    4926 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444811    4926 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444813    4926 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444816    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444818    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:28:23.446134 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444820    4926 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444823    4926 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444826    4926 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444828    4926 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444831    4926 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444833    4926 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444835    4926 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444838    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444841    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444845    4926 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444848    4926 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444851    4926 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444854    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444857    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444859    4926 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444862    4926 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444865    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444867    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444870    4926 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:28:23.446668 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444872    4926 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444875    4926 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444877    4926 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444880    4926 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444883    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444886    4926 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.444892    4926 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445699    4926 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445707    4926 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445710    4926 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445713    4926 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445716    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445719    4926 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445721    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445724    4926 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445727    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445729    4926 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445732    4926 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445734    4926 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:28:23.447132 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445737    4926 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445740    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445743    4926 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445746    4926 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445748    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445751    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445753    4926 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445756    4926 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445758    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445761    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445763    4926 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445766    4926 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445768    4926 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445770    4926 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445773    4926 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445775    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445778    4926 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445780    4926 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445784    4926 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445787    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:28:23.447601 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445790    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445792    4926 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445795    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445797    4926 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445800    4926 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445802    4926 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445805    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445808    4926 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445810    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445812    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445815    4926 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445818    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445820    4926 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445823    4926 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445825    4926 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445827    4926 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445830    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445832    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445836    4926 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445839    4926 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:28:23.448121 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445843    4926 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445846    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445849    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445851    4926 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445854    4926 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445857    4926 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445860    4926 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445862    4926 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445866    4926 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445870    4926 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445873    4926 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445876    4926 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445879    4926 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445882    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445884    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445886    4926 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445889    4926 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445891    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445903    4926 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445906    4926 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:28:23.448625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445909    4926 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445911    4926 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445913    4926 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445916    4926 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445919    4926 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445921    4926 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445924    4926 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445926    4926 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445929    4926 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445931    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445934    4926 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445936    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445939    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445941    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445944    4926 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445947    4926 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445949    4926 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445952    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445954    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445956    4926 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:28:23.449107 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445959    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445961    4926 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445964    4926 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445966    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445975    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445977    4926 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445980    4926 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445984    4926 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445986    4926 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445989    4926 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445991    4926 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445993    4926 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445996    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.445998    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446519    4926 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446526    4926 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446538    4926 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446541    4926 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446544    4926 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:28:23.449606 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446548    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446552    4926 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446556    4926 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446563    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446566    4926 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446569    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446571    4926 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446574    4926 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446576    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446579    4926 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446581    4926 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446584    4926 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446586    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446589    4926 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446591    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446594    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446596    4926 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446600    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446603    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:28:23.450062 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446606    4926 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446608    4926 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446611    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446614    4926 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446616    4926 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446618    4926 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446621    4926 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446624    4926 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446626    4926 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446628    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446631    4926 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446633    4926 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446636    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446638    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446641    4926 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446643    4926 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446647    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446650    4926 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446652    4926 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446655    4926 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:28:23.450543 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446657    4926 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446659    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446662    4926 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446664    4926 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446666    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446669    4926 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446672    4926 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446675    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446677    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446680    4926 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446682    4926 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446685    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446688    4926 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446690    4926 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446692    4926 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446695    4926 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446697    4926 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446700    4926 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446702    4926 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446704    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:28:23.451248 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446707    4926 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446709    4926 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446711    4926 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446714    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446716    4926 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446719    4926 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446721    4926 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446724    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446730    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446734    4926 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446738    4926 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446741    4926 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446745    4926 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446748    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446753    4926 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446757    4926 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446760    4926 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446763    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446765    4926 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446767    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:28:23.452088 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446770    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446772    4926 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446775    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446777    4926 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446780    4926 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446783    4926 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446785    4926 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446788    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446790    4926 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446795    4926 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446798    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446801    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446804    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446806    4926 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446809    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446811    4926 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446814    4926 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446816    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446820    4926 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:28:23.452619 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446823    4926 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446828    4926 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.446832    4926 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.446938    4926 flags.go:64] FLAG: --address="0.0.0.0"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.446956    4926 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.446963    4926 flags.go:64] FLAG: --anonymous-auth="true"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.446967    4926 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.446971    4926 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.446974    4926 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.446979    4926 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.446982    4926 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.446986    4926 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.446989    4926 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.446992    4926 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.446995    4926 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.446998    4926 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447002    4926 flags.go:64] FLAG: --cgroup-root=""
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447005    4926 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447009    4926 flags.go:64] FLAG: --client-ca-file=""
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447011    4926 flags.go:64] FLAG: --cloud-provider="external"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447014    4926 flags.go:64] FLAG: --cluster-dns="[]"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447022    4926 flags.go:64] FLAG: --cluster-domain=""
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447024    4926 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447028    4926 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 31 03:28:23.453099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447031    4926 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447034    4926 flags.go:64] FLAG: --container-log-max-files="5"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447038    4926 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447040    4926 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447044    4926 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447047    4926 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447050    4926 flags.go:64] FLAG: --contention-profiling="false"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447053    4926 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447056    4926 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447060    4926 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447063    4926 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447067    4926 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447071    4926 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447074    4926 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447077    4926 flags.go:64] FLAG: --enable-load-reader="false"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447080    4926 flags.go:64] FLAG: --enable-server="true"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447082    4926 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447086    4926 flags.go:64] FLAG: --event-burst="100"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447089    4926 flags.go:64] FLAG: --event-qps="50"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447092    4926 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447095    4926 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447098    4926 flags.go:64] FLAG: --eviction-hard=""
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447102    4926 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447105    4926 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447108    4926 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 31 03:28:23.453695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447111    4926 flags.go:64] FLAG: --eviction-soft=""
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447114    4926 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447117    4926 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447120    4926 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447123    4926 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447126    4926 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447129    4926 flags.go:64] FLAG: --fail-swap-on="true"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447132    4926 flags.go:64] FLAG: --feature-gates=""
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447136    4926 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447140    4926 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447143    4926 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447146    4926 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447149    4926 flags.go:64] FLAG: --healthz-port="10248"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447152    4926 flags.go:64] FLAG: --help="false"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447155    4926 flags.go:64] FLAG: --hostname-override="ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447159    4926 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447162    4926 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447165    4926 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447168    4926 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447172    4926 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447175    4926 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447177    4926 flags.go:64] FLAG: --image-service-endpoint=""
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447180    4926 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 31 03:28:23.454345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447183    4926 flags.go:64] FLAG: --kube-api-burst="100"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447185    4926 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447188    4926 flags.go:64] FLAG: --kube-api-qps="50"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447191    4926 flags.go:64] FLAG: --kube-reserved=""
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447194    4926 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447196    4926 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447199    4926 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447202    4926 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447205    4926 flags.go:64] FLAG: --lock-file=""
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447207    4926 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447210    4926 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447213    4926 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447218    4926 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447222    4926 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447225    4926 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447228    4926 flags.go:64] FLAG: --logging-format="text"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447230    4926 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447234    4926 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447236    4926 flags.go:64] FLAG: --manifest-url=""
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447239    4926 flags.go:64] FLAG: --manifest-url-header=""
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447243    4926 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447246    4926 flags.go:64] FLAG: --max-open-files="1000000"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447250    4926 flags.go:64] FLAG: --max-pods="110"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447252    4926 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447255    4926 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 31 03:28:23.454910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447258    4926 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447262    4926 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447282    4926 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447285    4926 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447288    4926 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447296    4926 flags.go:64] FLAG: --node-status-max-images="50"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447299    4926 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447302    4926 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447305    4926 flags.go:64] FLAG: --pod-cidr=""
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447308    4926 flags.go:64] FLAG: --pod-manifest-path=""
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447311    4926 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447313    4926 flags.go:64] FLAG: --pods-per-core="0"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447316    4926 flags.go:64] FLAG: --port="10250"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447319    4926 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447322    4926 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-02580f19870119d71"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447325    4926 flags.go:64] FLAG: --qos-reserved=""
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447327    4926 flags.go:64] FLAG: --read-only-port="10255"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447330    4926 flags.go:64] FLAG: --register-node="true"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447333    4926 flags.go:64] FLAG: --register-with-taints=""
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447337    4926 flags.go:64] FLAG: --registry-burst="10"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447341    4926 flags.go:64] FLAG: --registry-qps="5"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447344    4926 flags.go:64] FLAG: --reserved-cpus=""
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447347    4926 flags.go:64] FLAG: --reserved-memory=""
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447350    4926 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447354    4926 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447357    4926 flags.go:64] FLAG: --rotate-certificates="false"
Oct 31 03:28:23.455652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447359    4926 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447362    4926 flags.go:64] FLAG: --runonce="false"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447365    4926 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447368    4926 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447371    4926 flags.go:64] FLAG: --seccomp-default="false"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447374    4926 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447376    4926 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447379    4926 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447382    4926 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447385    4926 flags.go:64] FLAG: --storage-driver-password="root"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447387    4926 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447390    4926 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447393    4926 flags.go:64] FLAG: --storage-driver-user="root"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447396    4926 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447399    4926 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447401    4926 flags.go:64] FLAG: --system-cgroups=""
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447404    4926 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447409    4926 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447412    4926 flags.go:64] FLAG: --tls-cert-file=""
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447414    4926 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447418    4926 flags.go:64] FLAG: --tls-min-version=""
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447421    4926 flags.go:64] FLAG: --tls-private-key-file=""
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447424    4926 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447426    4926 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447429    4926 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 31 03:28:23.456307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447432    4926 flags.go:64] FLAG: --v="2"
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447440    4926 flags.go:64] FLAG: --version="false"
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447445    4926 flags.go:64] FLAG: --vmodule=""
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447449    4926 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447452    4926 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447575    4926 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447579    4926 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447581    4926 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447584    4926 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447586    4926 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447589    4926 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447591    4926 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447594    4926 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447596    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447599    4926 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447601    4926 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447604    4926 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447607    4926 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447609    4926 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447612    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447615    4926 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:28:23.456993 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447617    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447620    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447622    4926 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447624    4926 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447627    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447630    4926 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447632    4926 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447635    4926 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447639    4926 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447641    4926 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447643    4926 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447646    4926 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447649    4926 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447651    4926 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447654    4926 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447657    4926 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447659    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447662    4926 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447664    4926 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:28:23.457659 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447667    4926 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447669    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447672    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447674    4926 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447677    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447679    4926 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447681    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447684    4926 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447686    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447689    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447693    4926 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447696    4926 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447701    4926 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447704    4926 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447706    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447709    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447712    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447714    4926 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447717    4926 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:28:23.458198 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447719    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447722    4926 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447726    4926 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447729    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447731    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447734    4926 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447736    4926 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447740    4926 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447744    4926 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447747    4926 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447749    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447752    4926 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447754    4926 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447756    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447759    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447761    4926 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447764    4926 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447766    4926 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447768    4926 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:28:23.458708 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447771    4926 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447773    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447776    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447778    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447780    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447783    4926 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447786    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447789    4926 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447791    4926 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447793    4926 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447796    4926 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447798    4926 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447801    4926 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447803    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447806    4926 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447809    4926 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447812    4926 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447814    4926 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447816    4926 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447819    4926 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:28:23.459154 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447821    4926 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447824    4926 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447826    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447829    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447831    4926 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447834    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447837    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447839    4926 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447841    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447844    4926 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447846    4926 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447849    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447851    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.447876    4926 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447910    4926 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447914    4926 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:28:23.459769 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447916    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447919    4926 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447923    4926 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447925    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447928    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447930    4926 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447933    4926 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447935    4926 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447937    4926 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447940    4926 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447942    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447945    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447948    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447951    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447953    4926 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447956    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447959    4926 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447961    4926 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447963    4926 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:28:23.460227 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447966    4926 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447969    4926 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447971    4926 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447974    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447976    4926 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447979    4926 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447981    4926 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447984    4926 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447986    4926 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447989    4926 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447991    4926 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447993    4926 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447996    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.447998    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448001    4926 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448004    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448007    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448009    4926 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448012    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448015    4926 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:28:23.460811 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448017    4926 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448019    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448022    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448024    4926 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448027    4926 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448031    4926 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448033    4926 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448035    4926 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448038    4926 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448040    4926 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448043    4926 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448045    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448048    4926 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448050    4926 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448053    4926 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448056    4926 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448058    4926 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448061    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448063    4926 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448065    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:28:23.461376 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448068    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448070    4926 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448073    4926 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448075    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448078    4926 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448080    4926 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448083    4926 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448086    4926 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448089    4926 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448091    4926 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448094    4926 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448096    4926 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448099    4926 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448101    4926 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448104    4926 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448106    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448109    4926 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448112    4926 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448115    4926 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:28:23.461931 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448117    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448121    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448124    4926 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448126    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448129    4926 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448131    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448134    4926 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448136    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448140    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448144    4926 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448147    4926 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448149    4926 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448152    4926 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448154    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448157    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448159    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448162    4926 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448164    4926 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448167    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:28:23.462583 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448169    4926 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448173    4926 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448176    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448178    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448181    4926 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448183    4926 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.448188    4926 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.452811    4926 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.452827    4926 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.452847    4926 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.452851    4926 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452908    4926 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452913    4926 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452917    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452920    4926 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452922    4926 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452925    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452928    4926 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452930    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:28:23.463049 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452933    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452936    4926 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452939    4926 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452941    4926 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452944    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452946    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452949    4926 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452952    4926 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452954    4926 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452957    4926 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452959    4926 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452961    4926 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452964    4926 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452966    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452969    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452971    4926 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452974    4926 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452976    4926 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452981    4926 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:28:23.463521 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452985    4926 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452987    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452990    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452993    4926 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452996    4926 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.452998    4926 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453001    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453004    4926 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453006    4926 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453009    4926 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453011    4926 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453013    4926 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453016    4926 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453019    4926 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453022    4926 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453024    4926 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453027    4926 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453029    4926 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453032    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453035    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:28:23.463979 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453037    4926 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453040    4926 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453042    4926 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453045    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453047    4926 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453049    4926 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453052    4926 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453054    4926 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453057    4926 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453059    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453061    4926 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453064    4926 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453066    4926 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453069    4926 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453072    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453074    4926 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453077    4926 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453079    4926 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453082    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453084    4926 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:28:23.464481 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453086    4926 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453089    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453091    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453094    4926 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453096    4926 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453099    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453101    4926 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453104    4926 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453106    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453109    4926 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453112    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453114    4926 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453118    4926 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453120    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453123    4926 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453127    4926 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453130    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453133    4926 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453136    4926 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:28:23.464955 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453139    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453141    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453143    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453146    4926 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453148    4926 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453150    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453153    4926 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453155    4926 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453158    4926 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453160    4926 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453163    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453165    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453167    4926 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453170    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453172    4926 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453175    4926 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453177    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453180    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453182    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:28:23.465531 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453184    4926 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.453211    4926 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453363    4926 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453369    4926 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453373    4926 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453375    4926 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453378    4926 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453381    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453384    4926 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453386    4926 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453389    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453391    4926 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453394    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453396    4926 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453399    4926 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453401    4926 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:28:23.465990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453403    4926 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453406    4926 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453408    4926 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453411    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453413    4926 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453416    4926 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453418    4926 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453421    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453423    4926 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453425    4926 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453428    4926 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453430    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453433    4926 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453435    4926 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453438    4926 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453440    4926 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453442    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453445    4926 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453447    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:28:23.466387 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453450    4926 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453452    4926 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453455    4926 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453457    4926 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453460    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453463    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453466    4926 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453468    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453471    4926 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453473    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453476    4926 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453480    4926 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453483    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453486    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453489    4926 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453491    4926 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453493    4926 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453498    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453501    4926 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:28:23.467037 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453504    4926 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453506    4926 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453509    4926 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453511    4926 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453514    4926 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453516    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453519    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453521    4926 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453524    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453526    4926 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453529    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453531    4926 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453534    4926 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453536    4926 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453539    4926 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453541    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453545    4926 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453549    4926 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453552    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:28:23.467614 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453555    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453558    4926 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453561    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453564    4926 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453567    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453569    4926 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453572    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453574    4926 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453576    4926 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453579    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453581    4926 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453585    4926 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453597    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453600    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453603    4926 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453605    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453608    4926 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453616    4926 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453619    4926 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453622    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:28:23.468375 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453624    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453627    4926 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453629    4926 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453632    4926 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453634    4926 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453637    4926 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453640    4926 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453642    4926 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453645    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453647    4926 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453650    4926 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453652    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453654    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453657    4926 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453660    4926 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.453684    4926 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 31 03:28:23.469000 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453715    4926 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453719    4926 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453722    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453730    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453733    4926 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453735    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453738    4926 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453741    4926 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453744    4926 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453751    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453753    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453756    4926 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453759    4926 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453762    4926 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453764    4926 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453767    4926 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453770    4926 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453772    4926 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453775    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453777    4926 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 31 03:28:23.469584 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453779    4926 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453782    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453784    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453786    4926 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453789    4926 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453791    4926 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453794    4926 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453797    4926 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453807    4926 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453810    4926 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453812    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453815    4926 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453817    4926 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453820    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453822    4926 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453825    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453827    4926 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453830    4926 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453832    4926 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453834    4926 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 31 03:28:23.470240 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453837    4926 feature_gate.go:386] unrecognized feature gate: Example2
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453840    4926 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453842    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453845    4926 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453847    4926 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453850    4926 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453852    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453854    4926 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453857    4926 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453859    4926 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453862    4926 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453864    4926 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453867    4926 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453869    4926 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453871    4926 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453874    4926 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453876    4926 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453879    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453881    4926 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 31 03:28:23.470990 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453884    4926 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453886    4926 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453889    4926 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453891    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453895    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453898    4926 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453901    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453903    4926 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453905    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453909    4926 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453912    4926 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453915    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453918    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453921    4926 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453923    4926 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453926    4926 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453928    4926 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453930    4926 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453933    4926 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 31 03:28:23.471470 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453935    4926 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453938    4926 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453940    4926 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453942    4926 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453945    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453947    4926 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453949    4926 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453952    4926 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453955    4926 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453957    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453959    4926 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453962    4926 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453964    4926 feature_gate.go:386] unrecognized feature gate: Example
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453967    4926 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453970    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453973    4926 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453976    4926 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453979    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453982    4926 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 31 03:28:23.471989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453984    4926 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453987    4926 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453989    4926 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453992    4926 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453994    4926 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453997    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.453999    4926 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.454002    4926 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:23.454004    4926 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.454058    4926 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.455015    4926 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.455064    4926 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.455236    4926 server.go:1014] "Starting client certificate rotation"
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.455389    4926 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.455438    4926 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-30 03:22:41 +0000 UTC" deadline="2028-06-15 03:26:14.88933207 +0000 UTC"
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.455464    4926 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14231h57m51.433870166s"
Oct 31 03:28:23.472471 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.455697    4926 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 03:28:23.472891 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.456726    4926 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 31 03:28:23.472891 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.457786    4926 log.go:25] "Validated CRI v1 runtime API"
Oct 31 03:28:23.472891 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.459407    4926 log.go:25] "Validated CRI v1 image API"
Oct 31 03:28:23.472891 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.460149    4926 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 31 03:28:23.472891 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.461965    4926 fs.go:135] Filesystem UUIDs: map[50b1979b-bde4-46c8-94d6-889a96d173ee:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 fc3b5342-58de-4686-9cc0-de8f5220e968:/dev/nvme0n1p4]
Oct 31 03:28:23.473036 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.461986    4926 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/021c9211dacc25c411efcc28b88172db25c870666c45876af00c3e0b3afcd9d3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/021c9211dacc25c411efcc28b88172db25c870666c45876af00c3e0b3afcd9d3/userdata/shm major:0 minor:258 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/08de644521b27389f620f4e962dfa265629780ee93c9c43a23b0a689a3862814/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/08de644521b27389f620f4e962dfa265629780ee93c9c43a23b0a689a3862814/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2a270b7bcead4afcea718fce44060043418f249c2f45a3345e912124014faabc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2a270b7bcead4afcea718fce44060043418f249c2f45a3345e912124014faabc/userdata/shm major:0 minor:340 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/355e77d7d042a077b378b74001f86e9ac00bcb2b82e53610cfba4a2b7bdc98b6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/355e77d7d042a077b378b74001f86e9ac00bcb2b82e53610cfba4a2b7bdc98b6/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/52e93485e8131c2278d113ade1bc0d730f241150bc93a844897c00b87de722b8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/52e93485e8131c2278d113ade1bc0d730f241150bc93a844897c00b87de722b8/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/642584615ebce25cc91736eb61b391b4458f0fc1c67a4cc8b1f1194a58f4406a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/642584615ebce25cc91736eb61b391b4458f0fc1c67a4cc8b1f1194a58f4406a/userdata/shm major:0 minor:288 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/787c619e13f321ed8e26d02ae51fbcb9fefa5c47dae7feecddb0297c6cf5dc88/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/787c619e13f321ed8e26d02ae51fbcb9fefa5c47dae7feecddb0297c6cf5dc88/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/87f7f97822d72850958495720216553b16fb5554dba8399e677ebbede15c3681/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/87f7f97822d72850958495720216553b16fb5554dba8399e677ebbede15c3681/userdata/shm major:0 minor:220 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9bf724622b185f83fa250bf59ebd5ea37bec456e6b1d6336cc4bb00f11278944/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9bf724622b185f83fa250bf59ebd5ea37bec456e6b1d6336cc4bb00f11278944/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9e05680ab6472f21635513f010c0f8f119a622a9aaefe68e7131da9079199fe0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9e05680ab6472f21635513f010c0f8f119a622a9aaefe68e7131da9079199fe0/userdata/shm major:0 minor:211 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a26d728ea796615b99095298698a22f527fa1d67c9d2ca50ca1d973c156fde50/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a26d728ea796615b99095298698a22f527fa1d67c9d2ca50ca1d973c156fde50/userdata/shm major:0 minor:222 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a5c7d8bb069658a5c863248fbc65be1598745fe8dc4f185203058a3f55581490/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a5c7d8bb069658a5c863248fbc65be1598745fe8dc4f185203058a3f55581490/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a9b81ffeca9738d9b69e850c3c29b774f5e3c7b7923e97044f0388e1832daa88/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a9b81ffeca9738d9b69e850c3c29b774f5e3c7b7923e97044f0388e1832daa88/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bc01fe78c59ccb5472e9521b831780a720cf37ce4f1fdb34f65cdb0663490878/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bc01fe78c59ccb5472e9521b831780a720cf37ce4f1fdb34f65cdb0663490878/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bc146c9109a27614c348b38314d89af18695612cd24083d4005a28168fe9f068/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bc146c9109a27614c348b38314d89af18695612cd24083d4005a28168fe9f068/userdata/shm major:0 minor:229 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bc16d1c6f915a7bd3ac441e8c26b54e6c810a20251dfb14a7f53247f959ddb00/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bc16d1c6f915a7bd3ac441e8c26b54e6c810a20251dfb14a7f53247f959ddb00/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bd6e84050794860cc4853cd9383b8adbb6463bacc36a757dcf79c40ce29bf13d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bd6e84050794860cc4853cd9383b8adbb6463bacc36a757dcf79c40ce29bf13d/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ceba6febe347efdf3b0c58e1f0f45026529f478f8944770af62d2b507e27873e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ceba6febe347efdf3b0c58e1f0f45026529f478f8944770af62d2b507e27873e/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cf51b597e41d1cde6a03e33a56f068a39681f62798cb4a7a6e2ff78d8cd63f94/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cf51b597e41d1cde6a03e33a56f068a39681f62798cb4a7a6e2ff78d8cd63f94/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e096f86fb472ca5958f5b1079e2a53a7423eedc1a37fdc1c2df72f9989337a96/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e096f86fb472ca5958f5b1079e2a53a7423eedc1a37fdc1c2df72f9989337a96/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e49334cf60cfa11b1eebfe5d37557621069c99e7e5d28658af5ca201f7d8b9cb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e49334cf60cfa11b1eebfe5d37557621069c99e7e5d28658af5ca201f7d8b9cb/userdata/shm major:0 minor:290 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e820823f0827d4209635673a4e355c31477a5ed86dc6d50ad571e6ce46a7f151/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e820823f0827d4209635673a4e355c31477a5ed86dc6d50ad571e6ce46a7f151/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f3bb901b1e16080db3a21651bacc8711afaf29713df03fc9282ba14ab4583c77/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f3bb901b1e16080db3a21651bacc8711afaf29713df03fc9282ba14ab4583c77/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/02dec91e-45a1-4609-9b09-72ba5259bd0e/volumes/kubernetes.io~projected/kube-api-access-bdrgg:{mountpoint:/var/lib/kubelet/pods/02dec91e-45a1-4609-9b09-72ba5259bd0e/volumes/kubernetes.io~projected/kube-api-access-bdrgg major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/05c25677-4b67-4a79-bbca-25bcc3ef43f1/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/05c25677-4b67-4a79-bbca-25bcc3ef43f1/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/05c25677-4b67-4a79-bbca-25bcc3ef43f1/volumes/kubernetes.io~projected/kube-api-access-scqrb:{mountpoint:/var/lib/kubelet/pods/05c25677-4b67-4a79-bbca-25bcc3ef43f1/volumes/kubernetes.io~projected/kube-api-access-scqrb major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/05c25677-4b67-4a79-bbca-25bcc3ef43f1/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/05c25677-4b67-4a79-bbca-25bcc3ef43f1/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/05c25677-4b67-4a79-bbca-25bcc3ef43f1/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/05c25677-4b67-4a79-bbca-25bcc3ef43f1/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8/volumes/kubernetes.io~projected/kube-api-access-56fwn:{mountpoint:/var/lib/kubelet/pods/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8/volumes/kubernetes.io~projected/kube-api-access-56fwn major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bf0c577-0537-46fd-950a-ce5f565a51da/volumes/kubernetes.io~projected/kube-api-access-6kk5j:{mountpoint:/var/lib/kubelet/pods/0bf0c577-0537-46fd-950a-ce5f565a51da/volumes/kubernetes.io~projected/kube-api-access-6kk5j major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0bf0c577-0537-46fd-950a-ce5f565a51da/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/0bf0c577-0537-46fd-950a-ce5f565a51da/volumes/kubernetes.io~secret/serving-cert major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0d9f6de8-463e-4779-9825-e4318b9273b4/volumes/kubernetes.io~projected/kube-api-access-drh58:{mountpoint:/var/lib/kubelet/pods/0d9f6de8-463e-4779-9825-e4318b9273b4/volumes/kubernetes.io~projected/kube-api-access-drh58 major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/260390fb-c26d-4989-9e8a-a104c398e1c6/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/260390fb-c26d-4989-9e8a-a104c398e1c6/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/260390fb-c26d-4989-9e8a-a104c398e1c6/volumes/kubernetes.io~projected/kube-api-access-pfc5g:{mountpoint:/var/lib/kubelet/pods/260390fb-c26d-4989-9e8a-a104c398e1c6/volumes/kubernetes.io~projected/kube-api-access-pfc5g major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/260390fb-c26d-4989-9e8a-a104c398e1c6/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/260390fb-c26d-4989-9e8a-a104c398e1c6/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/260390fb-c26d-4989-9e8a-a104c398e1c6/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/260390fb-c26d-4989-9e8a-a104c398e1c6/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/26a79eda-be35-4aa8-b10b-cb2b1133a086/volumes/kubernetes.io~projected/kube-api-access-2l8rz:{mountpoint:/var/lib/kubelet/pods/26a79eda-be35-4aa8-b10b-cb2b1133a086/volumes/kubernetes.io~projected/kube-api-access-2l8rz major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/26a79eda-be35-4aa8-b10b-cb2b1133a086/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/26a79eda-be35-4aa8-b10b-cb2b1133a086/volumes/kubernetes.io~secret/default-certificate major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/26a79eda-be35-4aa8-b10b-cb2b1133a086/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/26a79eda-be35-4aa8-b10b-cb2b1133a086/volumes/kubernetes.io~secret/stats-auth major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3add72f4-7034-4186-95b5-59d2bd770827/volumes/kubernetes.io~projected/kube-api-access-tgprc:{mountpoint:/var/lib/kubelet/pods/3add72f4-7034-4186-95b5-59d2bd770827/volumes/kubernetes.io~projected/kube-api-access-tgprc major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4700dca5-f15a-4329-b719-828f5475810e/volumes/kubernetes.io~projected/kube-api-access-7svn9:{mountpoint:/var/lib/kubelet/pods/4700dca5-f15a-4329-b719-828f5475810e/volumes/kubernetes.io~projected/kube-api-access-7svn9 major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5615d56d-7e1b-4f88-803c-b6a1c0435263/volumes/kubernetes.io~projected/kube-api-access-b7j7n:{mountpoint:/var/lib/kubelet/pods/5615d56d-7e1b-4f88-803c-b6a1c0435263/volumes/kubernetes.io~projected/kube-api-access-b7j7n major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5f03b006-c2d0-4e6b-920b-88d146280084/volumes/kubernetes.io~projected/kube-api-access-7jdm6:{mountpoint:/var/lib/kubelet/pods/5f03b006-c2d0-4e6b-920b-88d146280084/volumes/kubernetes.io~projected/kube-api-access-7jdm6 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/609f4320-8b1b-459e-9e9b-9aeeed31dbe2/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/609f4320-8b1b-459e-9e9b-9aeeed31dbe2/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/65e4e478-1f58-498d-b6eb-619307c2943f/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/65e4e478-1f58-498d-b6eb-619307c2943f/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/65e4e478-1f58-498d-b6eb-619307c2943f/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/65e4e478-1f58-498d-b6eb-619307c2943f/volumes/kubernetes.io~empty-dir/tmp major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/65e4e478-1f58-498d-b6eb-619307c2943f/volumes/kubernetes.io~projected/kube-api-access-6ztgb:{mountpoint:/var/lib/kubelet/pods/65e4e478-1f58-498d-b6eb-619307c2943f/volumes/kubernetes.io~projected/kube-api-access-6ztgb major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/694e1b6d-1615-4134-b262-e9df58af99b5/volumes/kubernetes.io~projected/kube-api-access-s87l5:{mountpoint:/var/lib/kubelet/pods/694e1b6d-1615-4134-b262-e9df58af99b5/volumes/kubernetes.io~projected/kube-api-access-s87l5 major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7cc09f0c-627f-4445-b172-9556cab2bbdd/volumes/kubernetes.io~projected/kube-api-access-5js2v:{mountpoint:/var/lib/kubelet/pods/7cc09f0c-627f-4445-b172-9556cab2bbdd/volumes/kubernetes.io~projected/kube-api-access-5js2v major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7cc09f0c-627f-4445-b172-9556cab2bbdd/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/7cc09f0c-627f-4445-b172-9556cab2bbdd/volumes/kubernetes.io~secret/serving-cert major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ccdcc49-f072-4e15-b021-9bffc3fb674a/volumes/kubernetes.io~projected/kube-api-access-spbdb:{mountpoint:/var/lib/kubelet/pods/7ccdcc49-f072-4e15-b021-9bffc3fb674a/volumes/kubernetes.io~projected/kube-api-access-spbdb major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8d7cafa8-5ce8-4991-8598-5e03ebfb0b75/volumes/kubernetes.io~projected/kube-api-access-sklx7:{mountpoint:/var/lib/kubelet/pods/8d7cafa8-5ce8-4991-8598-5e03ebfb0b75/volumes/kubernetes.io~projected/kube-api-access-sklx7 major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9193a1f4-f4aa-4364-ae16-5520e88d8cc3/volumes/kubernetes.io~projected/kube-api-access-rfb57:{mountpoint:/var/lib/kubelet/pods/9193a1f4-f4aa-4364-ae16-5520e88d8cc3/volumes/kubernetes.io~projected/kube-api-access-rfb57 major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/945a916b-41c5-46f6-9c1a-9a8c93f7d6d2/volumes/kubernetes.io~projected/kube-api-access-h7wqg:{mountpoint:/var/lib/kubelet/pods/945a916b-41c5-46f6-9c1a-9a8c93f7d6d2/volumes/kubernetes.io~projected/kube-api-access-h7wqg major:0 minor:335 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a0883de4-ab87-4a25-bd8b-ecd4914d4638/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/a0883de4-ab87-4a25-bd8b-ecd4914d4638/volumes/kubernetes.io~secret/agent-certs major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a52638f4-7a5b-45de-9054-d173d66c7521/volumes/kubernetes.io~projected/kube-api-access-gcq98:{mountpoint:/var/lib/kubelet/pods/a52638f4-7a5b-45de-9054-d173d66c7521/volumes/kubernetes.io~projected/kube-api-access-gcq98 major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac8c7ff3-c156-4201-a297-ea6d1fd48e55/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/ac8c7ff3-c156-4201-a297-ea6d1fd48e55/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac8c7ff3-c156-4201-a297-ea6d1fd48e55/volumes/kubernetes.io~projected/kube-api-access-zm2wv:{mountpoint:/var/lib/kubelet/pods/ac8c7ff3-c156-4201-a297-ea6d1fd48e55/volumes/kubernetes.io~projected/kube-api-access-zm2wv major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac8c7ff3-c156-4201-a297-ea6d1fd48e55/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/ac8c7ff3-c156-4201-a297-ea6d1fd48e55/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aef16625-d836-42cb-bf49-176f80e73474/volumes/kubernetes.io~projected/kube-api-access-gxbxj:{mountpoint:/var/lib/kubelet/pods/aef16625-d836-42cb-bf49-176f80e73474/volumes/kubernetes.io~projected/kube-api-access-gxbxj major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bb5340b5-c677-40d1-a448-498561c62a17/volumes/kubernetes.io~projected/kube-api-access-549vb:{mountpoint:/var/lib/kubelet/pods/bb5340b5-c677-40d1-a448-498561c62a17/volumes/kubernetes.io~projected/kube-api-access-549vb major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bb5340b5-c677-40d1-a448-498561c62a17/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/bb5340b5-c677-40d1-a448-498561c62a17/volumes/kubernetes.io~secret/serving-cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bf18ddb4-723c-4954-8327-37f38b41eff1/volumes/kubernetes.io~projected/kube-api-access-j45vj:{mountpoint:/var/lib/kubelet/pods/bf18ddb4-723c-4954-8327-37f38b41eff1/volumes/kubernetes.io~projected/kube-api-access-j45vj major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a/volumes/kubernetes.io~projected/kube-api-access-pbg5x:{mountpoint:/var/lib/kubelet/pods/cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a/volumes/kubernetes.io~projected/kube-api-access-pbg5x major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cdbd24b7-07ab-44ba-a564-0b84466b7bfa/volumes/kubernetes.io~projected/kube-api-access-4r69c:{mountpoint:/var/lib/kubelet/pods/cdbd24b7-07ab-44ba-a564-0b84466b7bfa/volumes/kubernetes.io~projected/kube-api-access-4r69c major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d78ab9e2-f553-441e-9f3d-6d8985085c89/volumes/kubernetes.io~projected/kube-api-access-62xwv:{mountpoint:/var/lib/kubelet/pods/d78ab9e2-f553-441e-9f3d-6d8985085c89/volumes/kubernetes.io~projected/kube-api-access-62xwv major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/df73b930-8122-434b-b2a0-cc3dd2c44d5c/volumes/kubernetes.io~projected/kube-api-access-7dfdt:{mountpoint:/var/lib/kubelet/pods/df73b930-8122-434b-b2a0-cc3dd2c44d5c/volumes/kubernetes.io~projected/kube-api-access-7dfdt major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/df73b930-8122-434b-b2a0-cc3dd2c44d5c/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/df73b930-8122-434b-b2a0-cc3dd2c44d5c/volumes/kubernetes.io~secret/serving-cert major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a/volumes/kubernetes.io~projected/kube-api-access-c6q9r:{mountpoint:/var/lib/kubelet/pods/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a/volumes/kubernetes.io~projected/kube-api-access-c6q9r major:0 minor:237 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a/volumes/kubernetes.io~secret/cert major:0 minor:223 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/4eb5d354b253eda475c614fd18964ab2fe88e35c3464cc85d9924e2ed9f4f2cb/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/cce9d9f38ab682d0a62e61dcbc40fc4281478282b01bf4547c729956bc9975dd/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/295ef65bde679dd00674358bd5180e7e4dd970d39da048c386ffafe017a44fec/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/e501f3aa4f3413d922da5635bb2447c5c493fb622cfa532359d85fa46a034da4/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/030abc2186601e753f36f009dc96f8df929efba4f51e880e64a3bf96ff738a89/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/38e5e3e8c5b7424977f8d6221242e3fa09c22f0ff8c2f2844f1a87d309066f12/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/716a887310fb326b1f071b294bd18fe0a9cbe55b84483720b8e0682176326389/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/b0d3f26b0e64bc45e9f570b31241ab40da0ed76a06f7e0b725a188f98bd13c5a/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/03aaffaf31deb4867363b852c0085bac3006788782cbae9874c30aaabf49da84/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/0e8c508d607fe42bdbf3a90d1e107b3937dd70e3c122037ed2a17d649f546982/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/807976d813cc00d2c753e7012a378f60768f7e9cf0b2492db81ced907b92a337/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/0d660a9513d60e32e5209d6e6d6ee64b03d09562274c99837a0fa58c8da02aa0/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/010296aef39ea0281a22521e4a48522671d7f17f1771e83eb1bb2229e1a1b346/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/f03905d6556c3a77ca3230b36cfe75e8be0cca82e21c53b666b118489a3e9ccf/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/54c948573515ebcb53fb97dde53e49bdbc8ae4e10ced2c31546a6d6d1c115492/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/94fbce6311ad9dc91c461f0d4d7d3d148f9b4e3bcbac3a781695fec06c54ca7c/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/917cce79dda9c2c6f76b089bcd69aa6b1645d3142e2957bd78383fb9b6fbc5f9/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/7a24095db8e57b47f789d3d2834bd0fadd80428944dc5b80979f558c3f17d5e0/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/aa9b8281676b86965c78c70dccb543e9579aa24997515755a810324496a876ca/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/f1e6f17fcb52bfd91d6c42e06d6f110edfe9989821e0284f374a56f1bc7ae902/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/37e6b9ad2c2ddffb4d1b4fd3ffcac2c25f5bc7abbab7a0467834cf415fa28b12/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-246:{mountpoint:/var/lib/containers/storage/overlay/44e1a1a00cd74cc50a2f94f0d41dd00487d3d6cf5e622e27ecbc8ae733cec692/merged major:0 minor:246 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/f2052c1e2e5ad596f3c35fc3dd5f73efa581babb33729f8fcbe6ed8759b9f33a/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/be21822bf00ab2f08cb3111249b845295091ca0aafd156154a7d1abcb57bb5c3/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/d8108299839866a602370d6f8aaf8474f8c6f3f23ec8aafe8591c888bef67406/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/5daba1df1749a2525198232ff834b8f9fb7c7f197f690f77a16600f389899398/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/8c6e3e3a29f2281fd5a2a86bcfe3bb29b6aefc59a0808a932dbe4deb7f46b8ac/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/b238d0768f0ae1b09e8d238e5f657343ddf64992a39d0c8edb596012a2d5de3d/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/85e13d73cb08ee00cd330e7cc87631cac2291a1ed6e2d6f9dadf70090f31eb28/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/b3177f349b6a246df56bfaab1ed58a1cdbae52961d7d8e0d19f4899eb0b90b85/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-277:{mountpoint:/var/lib/containers/storage/overlay/05dd4bb2c4ddbba8d1a63fa159122011a3f7b3f4c0859c039f3288767e34a7eb/merged major:0 minor:277 fsType:overlay blockSize:0} overlay_0-279:{mountpoint:/var/lib/containers/storage/overlay/b543236beb2a29edbd6ce293d8215e34a787c5388fe46a5caaacc29d23f42d81/merged major:0 minor:279 fsType:overlay blockSize:0} overlay_0-281:{mountpoint:/var/lib/containers/storage/overlay/4b87cdbca7e4580b44fa1a067db650962a48507723dbbf9198f5ae04391ec9ba/merged major:0 minor:281 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/6b8b7ccf0dd4de02f8673b9354b3002afce0c3624759ef8e666a7399545c20b2/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-292:{mountpoint:/var/lib/containers/storage/overlay/87e0e585c43ad6fcc8a44502008f6b90b5108d686311fa8a4dd5c536452e1e79/merged major:0 minor:292 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/2d61b3d2d2710efd59eec98526a8f436360ac4d77545e3ed333fde0f47a88a27/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/a679a5a70dd16d74a53281feebebb6d23967d3f4c7ce041a2b266cf8bad5c800/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/5f86734c7e9be3973a2f5d2f701dfce2cfc0b59f788c5bebc6e6c7ca5ae6c858/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/bd39f7a5dbb077d30ce732071a2e4b3fb6067fc76ea52650cac189f15c67fc02/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/fa7976d79b28ceb379a0e0766940310362841e8996ba2374d185221f1d171749/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/e6b19d9c001f93efd22e3da7a9568c2795952b171d9e6d8adf01fe9d4cfee476/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-327:{mountpoint:/var/lib/containers/storage/overlay/7a50bab7ebc469a55b47a43f45cb720f4952ecb11b4f05955a2cb4bec3e2d171/merged major:0 minor:327 fsType:overlay blockSize:0} overlay_0-329:{mountpoint:/var/lib/containers/storage/overlay/ef403e7d469b23fa6c9b29d32c3d027c03f8ff1d1c156614d7e7e90b1d110747/merged major:0 minor:329 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/1354091924c4f278bf73010ceaad1ba4823de078e888f6fe1670ffbd127e26b1/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/e2ddee51eb0937522db36aa0e9e968d4ac8d0fec0fb56576e3c79a295adb6bdb/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/4c50b015231e7ff50576f113db669a2249bacbe75d0bb9fa9de295f8441e69de/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/3c3c4547cb9482f4d54f9c33323cae5817e4e73b71d47acfaeefee0f16dfe78f/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/01cc8490540c348f411b3b6f94880d03ad99b74209a0ffc00be37520cd1a0672/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/e1a306377c45b43ec2c1a9c7ceac2c52f8f8e0b83c40f0e508ea548825431be0/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/7461653303da855ab681d6cb4c8268ce7b6f0b7174a1bf70af80b9bbdfdb9d5e/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/d9adc089dac40bb02ec4f16a55eb9aa5f31c452fdb655e9ab308c0014fcbc671/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/982e8f566c8a055c1295e05fb054148f35fafd17c8261db0a961260285a7aa00/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/72beae036b33882789d939195527bc1f5b95412754f94178d92ac2ee8ea50869/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/d46c620a7e7a516bb086423f0658523e3362eaf81ac1e42c4b5f2ef086608c4f/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/d51f775b8e5ba29f8346fcfe8b2f745848ce5e4af85c0811aad13affb7dfa3f9/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.471974    4926 manager.go:217] Machine: {Timestamp:2026-10-31 03:28:23.4709853 +0000 UTC m=+0.056525553 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3089514 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f00b6779b2a329a67f9893d000f1d SystemUUID:ec2f00b6-779b-2a32-9a67-f9893d000f1d BootID:b5920995-388f-4b1b-9ff3-a7ada365d64b Filesystems:[{Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/609f4320-8b1b-459e-9e9b-9aeeed31dbe2/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:286 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bf18ddb4-723c-4954-8327-37f38b41eff1/volumes/kubernetes.io~projected/kube-api-access-j45vj DeviceMajor:0 DeviceMinor:202 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/cdbd24b7-07ab-44ba-a564-0b84466b7bfa/volumes/kubernetes.io~projected/kube-api-access-4r69c DeviceMajor:0 DeviceMinor:217 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/260390fb-c26d-4989-9e8a-a104c398e1c6/volumes/kubernetes.io~projected/kube-api-access-pfc5g DeviceMajor:0 DeviceMinor:233 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-246 DeviceMajor:0 DeviceMinor:246 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0bf0c577-0537-46fd-950a-ce5f565a51da/volumes/kubernetes.io~projected/kube-api-access-6kk5j DeviceMajor:0 DeviceMinor:232 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bc01fe78c59ccb5472e9521b831780a720cf37ce4f1fdb34f65cdb0663490878/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-292 DeviceMajor:0 DeviceMinor:292 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a0883de4-ab87-4a25-bd8b-ecd4914d4638/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:50 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ac8c7ff3-c156-4201-a297-ea6d1fd48e55/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/05c25677-4b67-4a79-bbca-25bcc3ef43f1/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:207 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bd6e84050794860cc4853cd9383b8adbb6463bacc36a757dcf79c40ce29bf13d/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/642584615ebce25cc91736eb61b391b4458f0fc1c67a4cc8b1f1194a58f4406a/userdata/shm DeviceMajor:0 DeviceMinor:288 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bc146c9109a27614c348b38314d89af18695612cd24083d4005a28168fe9f068/userdata/shm DeviceMajor:0 DeviceMinor:229 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/355e77d7d042a077b378b74001f86e9ac00bcb2b82e53610cfba4a2b7bdc98b6/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a/volumes/kubernetes.io~projected/kube-api-access-pbg5x DeviceMajor:0 DeviceMinor:197 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/26a79eda-be35-4aa8-b10b-cb2b1133a086/volumes/kubernetes.io~projected/kube-api-access-2l8rz DeviceMajor:0 DeviceMinor:219 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0bf0c577-0537-46fd-950a-ce5f565a51da/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:224 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9bf724622b185f83fa250bf59ebd5ea37bec456e6b1d6336cc4bb00f11278944/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/df73b930-8122-434b-b2a0-cc3dd2c44d5c/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:204 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/05c25677-4b67-4a79-bbca-25bcc3ef43f1/volumes/kubernetes.io~projected/kube-api-access-scqrb DeviceMajor:0 DeviceMinor:215 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/65e4e478-1f58-498d-b6eb-619307c2943f/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:48 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bc16d1c6f915a7bd3ac441e8c26b54e6c810a20251dfb14a7f53247f959ddb00/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-279 DeviceMajor:0 DeviceMinor:279 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/08de644521b27389f620f4e962dfa265629780ee93c9c43a23b0a689a3862814/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5615d56d-7e1b-4f88-803c-b6a1c0435263/volumes/kubernetes.io~projected/kube-api-access-b7j7n DeviceMajor:0 DeviceMinor:322 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e820823f0827d4209635673a4e355c31477a5ed86dc6d50ad571e6ce46a7f151/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ceba6febe347efdf3b0c58e1f0f45026529f478f8944770af62d2b507e27873e/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/02dec91e-45a1-4609-9b09-72ba5259bd0e/volumes/kubernetes.io~projected/kube-api-access-bdrgg DeviceMajor:0 DeviceMinor:45 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a5c7d8bb069658a5c863248fbc65be1598745fe8dc4f185203058a3f55581490/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/260390fb-c26d-4989-9e8a-a104c398e1c6/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:228 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8/volumes/kubernetes.io~projected/kube-api-access-56fwn DeviceMajor:0 DeviceMinor:287 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-329 DeviceMajor:0 DeviceMinor:329 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e096f86fb472ca5958f5b1079e2a53a7423eedc1a37fdc1c2df72f9989337a96/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/52e93485e8131c2278d113ade1bc0d730f241150bc93a844897c00b87de722b8/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/87f7f97822d72850958495720216553b16fb5554dba8399e677ebbede15c3681/userdata/shm DeviceMajor:0 DeviceMinor:220 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-327 DeviceMajor:0 DeviceMinor:327 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bb5340b5-c677-40d1-a448-498561c62a17/volumes/kubernetes.io~projected/kube-api-access-549vb DeviceMajor:0 DeviceMinor:213 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/260390fb-c26d-4989-9e8a-a104c398e1c6/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:226 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-277 DeviceMajor:0 DeviceMinor:277 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/05c25677-4b67-4a79-bbca-25bcc3ef43f1/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:206 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0d9f6de8-463e-4779-9825-e4318b9273b4/volumes/kubernetes.io~projected/kube-api-access-drh58 DeviceMajor:0 DeviceMinor:214 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e49334cf60cfa11b1eebfe5d37557621069c99e7e5d28658af5ca201f7d8b9cb/userdata/shm DeviceMajor:0 DeviceMinor:290 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cf51b597e41d1cde6a03e33a56f068a39681f62798cb4a7a6e2ff78d8cd63f94/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7cc09f0c-627f-4445-b172-9556cab2bbdd/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:227 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ac8c7ff3-c156-4201-a297-ea6d1fd48e55/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:47 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/021c9211dacc25c411efcc28b88172db25c870666c45876af00c3e0b3afcd9d3/userdata/shm DeviceMajor:0 DeviceMinor:258 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/65e4e478-1f58-498d-b6eb-619307c2943f/volumes/kubernetes.io~projected/kube-api-access-6ztgb DeviceMajor:0 DeviceMinor:55 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-281 DeviceMajor:0 DeviceMinor:281 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/65e4e478-1f58-498d-b6eb-619307c2943f/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:49 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9193a1f4-f4aa-4364-ae16-5520e88d8cc3/volumes/kubernetes.io~projected/kube-api-access-rfb57 DeviceMajor:0 DeviceMinor:53 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7ccdcc49-f072-4e15-b021-9bffc3fb674a/volumes/kubernetes.io~projected/kube-api-access-spbdb DeviceMajor:0 DeviceMinor:58 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bb5340b5-c677-40d1-a448-498561c62a17/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:203 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/3add72f4-7034-4186-95b5-59d2bd770827/volumes/kubernetes.io~projected/kube-api-access-tgprc DeviceMajor:0 DeviceMinor:57 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/5f03b006-c2d0-4e6b-920b-88d146280084/volumes/kubernetes.io~projected/kube-api-access-7jdm6 DeviceMajor:0 DeviceMinor:59 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a26d728ea796615b99095298698a22f527fa1d67c9d2ca50ca1d973c156fde50/userdata/shm DeviceMajor:0 DeviceMinor:222 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/df73b930-8122-434b-b2a0-cc3dd2c44d5c/volumes/kubernetes.io~projected/kube-api-access-7dfdt DeviceMajor:0 DeviceMinor:216 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7cc09f0c-627f-4445-b172-9556cab2bbdd/volumes/kubernetes.io~projected/kube-api-access-5js2v DeviceMajor:0 DeviceMinor:235 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/26a79eda-be35-4aa8-b10b-cb2b1133a086/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:205 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/694e1b6d-1615-4134-b262-e9df58af99b5/volumes/kubernetes.io~projected/kube-api-access-s87l5 DeviceMajor:0 DeviceMinor:54 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a9b81ffeca9738d9b69e850c3c29b774f5e3c7b7923e97044f0388e1832daa88/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a/volumes/kubernetes.io~projected/kube-api-access-c6q9r DeviceMajor:0 DeviceMinor:237 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/26a79eda-be35-4aa8-b10b-cb2b1133a086/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:208 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:223 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/260390fb-c26d-4989-9e8a-a104c398e1c6/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:231 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/8d7cafa8-5ce8-4991-8598-5e03ebfb0b75/volumes/kubernetes.io~projected/kube-api-access-sklx7 DeviceMajor:0 DeviceMinor:236 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2a270b7bcead4afcea718fce44060043418f249c2f45a3345e912124014faabc/userdata/shm DeviceMajor:0 DeviceMinor:340 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aef16625-d836-42cb-bf49-176f80e73474/volumes/kubernetes.io~projected/kube-api-access-gxbxj DeviceMajor:0 DeviceMinor:201 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9e05680ab6472f21635513f010c0f8f119a622a9aaefe68e7131da9079199fe0/userdata/shm DeviceMajor:0 DeviceMinor:211 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ac8c7ff3-c156-4201-a297-ea6d1fd48e55/volumes/kubernetes.io~projected/kube-api-access-zm2wv DeviceMajor:0 DeviceMinor:51 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4700dca5-f15a-4329-b719-828f5475810e/volumes/kubernetes.io~projected/kube-api-access-7svn9 DeviceMajor:0 DeviceMinor:56 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f3bb901b1e16080db3a21651bacc8711afaf29713df03fc9282ba14ab4583c77/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/05c25677-4b67-4a79-bbca-25bcc3ef43f1/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:218 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/945a916b-41c5-46f6-9c1a-9a8c93f7d6d2/volumes/kubernetes.io~projected/kube-api-access-h7wqg DeviceMajor:0 DeviceMinor:335 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a52638f4-7a5b-45de-9054-d173d66c7521/volumes/kubernetes.io~projected/kube-api-access-gcq98 DeviceMajor:0 DeviceMinor:234 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d78ab9e2-f553-441e-9f3d-6d8985085c89/volumes/kubernetes.io~projected/kube-api-access-62xwv DeviceMajor:0 DeviceMinor:52 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/787c619e13f321ed8e26d02ae51fbcb9fefa5c47dae7feecddb0297c6cf5dc88/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:021c9211dacc25c MacAddress:6a:f8:4b:bc:da:93 Speed:10000 Mtu:8901} {Name:2a270b7bcead4af MacAddress:aa:c2:4a:25:5e:14 Speed:10000 Mtu:8901} {Name:52e93485e8131c2 MacAddress:c6:3f:49:87:ce:a0 Speed:10000 Mtu:8901} {Name:642584615ebce25 MacAddress:c2:37:4e:28:d1:c2 Speed:10000 Mtu:8901} {Name:87f7f97822d7285 MacAddress:56:7b:49:f8:04:b9 Speed:10000 Mtu:8901} {Name:9bf724622b185f8 MacAddress:0a:27:c5:db:76:2e Speed:10000 Mtu:8901} {Name:9e05680ab6472f2 MacAddress:1a:b7:f2:05:84:cb Speed:10000 Mtu:8901} {Name:a26d728ea796615 MacAddress:b2:75:35:6b:ed:8b Speed:10000 Mtu:8901} {Name:a5c7d8bb069658a MacAddress:3a:5c:b8:65:0e:9a Speed:10000 Mtu:8901} {Name:bc01fe78c59ccb5 MacAddress:b2:18:c3:dd:7a:02 Speed:10000 Mtu:8901} {Name:bc146c9109a2761 MacAddress:6e:1e:7d:11:86:66 Speed:10000 Mtu:8901} {Name:bd6e84050794860 MacAddress:ee:54:46:16:03:f8 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:d7:57:ce:25 Speed:0 Mtu:9001} {Name:br-int MacAddress:b2:1d:24:8c:78:75 Speed:0 Mtu:8901} {Name:e49334cf60cfa11 MacAddress:46:b6:65:ff:e4:e5 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:d7:57:ce:25 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:86:1c:b7:7b:23:3e Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:ea:a0:14:52:5b:4f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.472338    4926 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.472381    4926 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.472632    4926 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.472661    4926 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-19.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.472779    4926 topology_manager.go:143] "Creating topology manager with none policy"
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.472786    4926 container_manager_linux.go:311] "Creating device plugin manager"
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.472797    4926 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.472817    4926 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.472835    4926 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.472978    4926 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.473071    4926 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.473103    4926 kubelet.go:493] "Attempting to sync node with API server"
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.473112    4926 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.473122    4926 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.473130    4926 kubelet.go:405] "Adding apiserver pod source"
Oct 31 03:28:23.473361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.473137    4926 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 31 03:28:23.474367 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.473788    4926 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 03:28:23.474367 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.473797    4926 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 03:28:23.474367 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.474216    4926 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 31 03:28:23.485258 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485242    4926 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 31 03:28:23.485258 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485256    4926 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 31 03:28:23.485382 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485370    4926 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 31 03:28:23.485553 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485543    4926 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 31 03:28:23.485590 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485571    4926 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 31 03:28:23.485664 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485655    4926 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 31 03:28:23.485699 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485670    4926 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 31 03:28:23.485699 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485676    4926 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 31 03:28:23.485699 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485681    4926 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 31 03:28:23.485699 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485687    4926 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 31 03:28:23.485699 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485692    4926 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 31 03:28:23.485699 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485698    4926 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 31 03:28:23.485866 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485703    4926 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 31 03:28:23.485866 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485713    4926 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 31 03:28:23.485866 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485722    4926 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 31 03:28:23.485866 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485737    4926 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 31 03:28:23.485866 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485746    4926 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 31 03:28:23.485866 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485781    4926 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 31 03:28:23.485866 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.485791    4926 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 31 03:28:23.489422 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.489408    4926 server.go:1263] "Started kubelet"
Oct 31 03:28:23.489809 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.489503    4926 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 31 03:28:23.489971 ip-10-0-130-19 systemd[1]: Started Kubernetes Kubelet.
Oct 31 03:28:23.490161 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.490025    4926 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 31 03:28:23.490161 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.490105    4926 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 31 03:28:23.490579 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.490402    4926 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 31 03:28:23.492073 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.492047    4926 server.go:317] "Adding debug handlers to kubelet server"
Oct 31 03:28:23.494548 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.494468    4926 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 31 03:28:23.494548 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.494494    4926 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 31 03:28:23.494681 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.494556    4926 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-30 03:22:42 +0000 UTC" deadline="2028-04-14 21:10:37.869479117 +0000 UTC"
Oct 31 03:28:23.494681 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.494574    4926 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12761h42m14.374908215s"
Oct 31 03:28:23.494681 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.494604    4926 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 31 03:28:23.494681 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.494615    4926 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 31 03:28:23.494898 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.494699    4926 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 31 03:28:23.495054 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:23.495030    4926 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-19.us-west-2.compute.internal\" not found"
Oct 31 03:28:23.496584 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.495968    4926 factory.go:55] Registering systemd factory
Oct 31 03:28:23.496584 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.495990    4926 factory.go:223] Registration of the systemd container factory successfully
Oct 31 03:28:23.497012 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.496686    4926 factory.go:153] Registering CRI-O factory
Oct 31 03:28:23.497012 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.496702    4926 factory.go:223] Registration of the crio container factory successfully
Oct 31 03:28:23.497012 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.496763    4926 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 31 03:28:23.497012 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.496782    4926 factory.go:103] Registering Raw factory
Oct 31 03:28:23.497012 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.496796    4926 manager.go:1199] Started watching for new ooms in manager
Oct 31 03:28:23.497306 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497128    4926 manager.go:319] Starting recovery of all containers
Oct 31 03:28:23.497306 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497233    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bf0c577-0537-46fd-950a-ce5f565a51da" volumeName="kubernetes.io/projected/0bf0c577-0537-46fd-950a-ce5f565a51da-kube-api-access-6kk5j" seLinuxMountContext=""
Oct 31 03:28:23.497306 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497282    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="694e1b6d-1615-4134-b262-e9df58af99b5" volumeName="kubernetes.io/empty-dir/694e1b6d-1615-4134-b262-e9df58af99b5-tmp-dir" seLinuxMountContext=""
Oct 31 03:28:23.497306 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497296    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb5340b5-c677-40d1-a448-498561c62a17" volumeName="kubernetes.io/configmap/bb5340b5-c677-40d1-a448-498561c62a17-config" seLinuxMountContext=""
Oct 31 03:28:23.497505 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497311    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="260390fb-c26d-4989-9e8a-a104c398e1c6" volumeName="kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-trusted-ca" seLinuxMountContext=""
Oct 31 03:28:23.497505 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497323    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26a79eda-be35-4aa8-b10b-cb2b1133a086" volumeName="kubernetes.io/projected/26a79eda-be35-4aa8-b10b-cb2b1133a086-kube-api-access-2l8rz" seLinuxMountContext=""
Oct 31 03:28:23.497505 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497335    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a52638f4-7a5b-45de-9054-d173d66c7521" volumeName="kubernetes.io/projected/a52638f4-7a5b-45de-9054-d173d66c7521-kube-api-access-gcq98" seLinuxMountContext=""
Oct 31 03:28:23.497505 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497346    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aef16625-d836-42cb-bf49-176f80e73474" volumeName="kubernetes.io/projected/aef16625-d836-42cb-bf49-176f80e73474-kube-api-access-gxbxj" seLinuxMountContext=""
Oct 31 03:28:23.497505 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497357    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc6215a3-f0b6-4f89-9a0d-30589bb43c5a" volumeName="kubernetes.io/secret/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a-cert" seLinuxMountContext=""
Oct 31 03:28:23.497505 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497373    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05c25677-4b67-4a79-bbca-25bcc3ef43f1" volumeName="kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-trusted-ca" seLinuxMountContext=""
Oct 31 03:28:23.497505 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497385    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26a79eda-be35-4aa8-b10b-cb2b1133a086" volumeName="kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-default-certificate" seLinuxMountContext=""
Oct 31 03:28:23.497505 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497397    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4700dca5-f15a-4329-b719-828f5475810e" volumeName="kubernetes.io/projected/4700dca5-f15a-4329-b719-828f5475810e-kube-api-access-7svn9" seLinuxMountContext=""
Oct 31 03:28:23.497505 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497408    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a52638f4-7a5b-45de-9054-d173d66c7521" volumeName="kubernetes.io/empty-dir/a52638f4-7a5b-45de-9054-d173d66c7521-tmp-dir" seLinuxMountContext=""
Oct 31 03:28:23.497505 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497421    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a" volumeName="kubernetes.io/projected/cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a-kube-api-access-pbg5x" seLinuxMountContext=""
Oct 31 03:28:23.497505 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497472    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df73b930-8122-434b-b2a0-cc3dd2c44d5c" volumeName="kubernetes.io/configmap/df73b930-8122-434b-b2a0-cc3dd2c44d5c-trusted-ca-bundle" seLinuxMountContext=""
Oct 31 03:28:23.497505 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497490    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fc6215a3-f0b6-4f89-9a0d-30589bb43c5a" volumeName="kubernetes.io/projected/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a-kube-api-access-c6q9r" seLinuxMountContext=""
Oct 31 03:28:23.497505 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497503    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05c25677-4b67-4a79-bbca-25bcc3ef43f1" volumeName="kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-installation-pull-secrets" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497517    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65e4e478-1f58-498d-b6eb-619307c2943f" volumeName="kubernetes.io/projected/65e4e478-1f58-498d-b6eb-619307c2943f-kube-api-access-6ztgb" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497529    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac8c7ff3-c156-4201-a297-ea6d1fd48e55" volumeName="kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovnkube-config" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497541    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df73b930-8122-434b-b2a0-cc3dd2c44d5c" volumeName="kubernetes.io/empty-dir/df73b930-8122-434b-b2a0-cc3dd2c44d5c-snapshots" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497553    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4700dca5-f15a-4329-b719-828f5475810e" volumeName="kubernetes.io/configmap/4700dca5-f15a-4329-b719-828f5475810e-iptables-alerter-script" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497564    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cc09f0c-627f-4445-b172-9556cab2bbdd" volumeName="kubernetes.io/secret/7cc09f0c-627f-4445-b172-9556cab2bbdd-serving-cert" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497578    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a0883de4-ab87-4a25-bd8b-ecd4914d4638" volumeName="kubernetes.io/secret/a0883de4-ab87-4a25-bd8b-ecd4914d4638-agent-certs" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497589    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb5340b5-c677-40d1-a448-498561c62a17" volumeName="kubernetes.io/secret/bb5340b5-c677-40d1-a448-498561c62a17-serving-cert" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497600    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bf0c577-0537-46fd-950a-ce5f565a51da" volumeName="kubernetes.io/secret/0bf0c577-0537-46fd-950a-ce5f565a51da-serving-cert" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497612    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65e4e478-1f58-498d-b6eb-619307c2943f" volumeName="kubernetes.io/empty-dir/65e4e478-1f58-498d-b6eb-619307c2943f-tmp" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497624    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cc09f0c-627f-4445-b172-9556cab2bbdd" volumeName="kubernetes.io/projected/7cc09f0c-627f-4445-b172-9556cab2bbdd-kube-api-access-5js2v" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497637    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" volumeName="kubernetes.io/projected/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-kube-api-access-rfb57" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497650    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac8c7ff3-c156-4201-a297-ea6d1fd48e55" volumeName="kubernetes.io/secret/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497691    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb5340b5-c677-40d1-a448-498561c62a17" volumeName="kubernetes.io/projected/bb5340b5-c677-40d1-a448-498561c62a17-kube-api-access-549vb" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497704    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cdbd24b7-07ab-44ba-a564-0b84466b7bfa" volumeName="kubernetes.io/projected/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-kube-api-access-4r69c" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497717    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df73b930-8122-434b-b2a0-cc3dd2c44d5c" volumeName="kubernetes.io/empty-dir/df73b930-8122-434b-b2a0-cc3dd2c44d5c-tmp" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497729    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8" volumeName="kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497740    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="260390fb-c26d-4989-9e8a-a104c398e1c6" volumeName="kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-installation-pull-secrets" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497752    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26a79eda-be35-4aa8-b10b-cb2b1133a086" volumeName="kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-stats-auth" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497764    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aef16625-d836-42cb-bf49-176f80e73474" volumeName="kubernetes.io/configmap/aef16625-d836-42cb-bf49-176f80e73474-telemetry-config" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497776    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df73b930-8122-434b-b2a0-cc3dd2c44d5c" volumeName="kubernetes.io/projected/df73b930-8122-434b-b2a0-cc3dd2c44d5c-kube-api-access-7dfdt" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497788    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ccdcc49-f072-4e15-b021-9bffc3fb674a" volumeName="kubernetes.io/configmap/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-daemon-config" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497801    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a0883de4-ab87-4a25-bd8b-ecd4914d4638" volumeName="kubernetes.io/configmap/a0883de4-ab87-4a25-bd8b-ecd4914d4638-konnectivity-ca" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497813    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02dec91e-45a1-4609-9b09-72ba5259bd0e" volumeName="kubernetes.io/projected/02dec91e-45a1-4609-9b09-72ba5259bd0e-kube-api-access-bdrgg" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497826    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05c25677-4b67-4a79-bbca-25bcc3ef43f1" volumeName="kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-certificates" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497837    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05c25677-4b67-4a79-bbca-25bcc3ef43f1" volumeName="kubernetes.io/empty-dir/05c25677-4b67-4a79-bbca-25bcc3ef43f1-ca-trust-extracted" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497850    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5615d56d-7e1b-4f88-803c-b6a1c0435263" volumeName="kubernetes.io/empty-dir/5615d56d-7e1b-4f88-803c-b6a1c0435263-data-volume" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497878    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f03b006-c2d0-4e6b-920b-88d146280084" volumeName="kubernetes.io/projected/5f03b006-c2d0-4e6b-920b-88d146280084-kube-api-access-7jdm6" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497890    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05c25677-4b67-4a79-bbca-25bcc3ef43f1" volumeName="kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-kube-api-access-scqrb" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497902    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05c25677-4b67-4a79-bbca-25bcc3ef43f1" volumeName="kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-image-registry-private-configuration" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497915    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a52638f4-7a5b-45de-9054-d173d66c7521" volumeName="kubernetes.io/configmap/a52638f4-7a5b-45de-9054-d173d66c7521-config-volume" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497930    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df73b930-8122-434b-b2a0-cc3dd2c44d5c" volumeName="kubernetes.io/configmap/df73b930-8122-434b-b2a0-cc3dd2c44d5c-service-ca-bundle" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497941    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="df73b930-8122-434b-b2a0-cc3dd2c44d5c" volumeName="kubernetes.io/secret/df73b930-8122-434b-b2a0-cc3dd2c44d5c-serving-cert" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.497988    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="260390fb-c26d-4989-9e8a-a104c398e1c6" volumeName="kubernetes.io/empty-dir/260390fb-c26d-4989-9e8a-a104c398e1c6-ca-trust-extracted" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498003    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f03b006-c2d0-4e6b-920b-88d146280084" volumeName="kubernetes.io/configmap/5f03b006-c2d0-4e6b-920b-88d146280084-serviceca" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498015    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ccdcc49-f072-4e15-b021-9bffc3fb674a" volumeName="kubernetes.io/configmap/7ccdcc49-f072-4e15-b021-9bffc3fb674a-cni-binary-copy" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498030    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" volumeName="kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cni-binary-copy" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498045    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="945a916b-41c5-46f6-9c1a-9a8c93f7d6d2" volumeName="kubernetes.io/projected/945a916b-41c5-46f6-9c1a-9a8c93f7d6d2-kube-api-access-h7wqg" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498058    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0bf0c577-0537-46fd-950a-ce5f565a51da" volumeName="kubernetes.io/configmap/0bf0c577-0537-46fd-950a-ce5f565a51da-config" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498071    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="260390fb-c26d-4989-9e8a-a104c398e1c6" volumeName="kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-kube-api-access-pfc5g" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498084    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="260390fb-c26d-4989-9e8a-a104c398e1c6" volumeName="kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-bound-sa-token" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498108    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3add72f4-7034-4186-95b5-59d2bd770827" volumeName="kubernetes.io/projected/3add72f4-7034-4186-95b5-59d2bd770827-kube-api-access-tgprc" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498122    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8d7cafa8-5ce8-4991-8598-5e03ebfb0b75" volumeName="kubernetes.io/projected/8d7cafa8-5ce8-4991-8598-5e03ebfb0b75-kube-api-access-sklx7" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498134    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cdbd24b7-07ab-44ba-a564-0b84466b7bfa" volumeName="kubernetes.io/configmap/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-config" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498146    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f4b87fa6-eceb-4783-b2c5-4685e08ba9ef" volumeName="kubernetes.io/configmap/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-nginx-conf" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498170    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" volumeName="kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498183    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac8c7ff3-c156-4201-a297-ea6d1fd48e55" volumeName="kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-env-overrides" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498194    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb5340b5-c677-40d1-a448-498561c62a17" volumeName="kubernetes.io/configmap/bb5340b5-c677-40d1-a448-498561c62a17-trusted-ca" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498206    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d9f6de8-463e-4779-9825-e4318b9273b4" volumeName="kubernetes.io/projected/0d9f6de8-463e-4779-9825-e4318b9273b4-kube-api-access-drh58" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498217    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="260390fb-c26d-4989-9e8a-a104c398e1c6" volumeName="kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-certificates" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498229    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5615d56d-7e1b-4f88-803c-b6a1c0435263" volumeName="kubernetes.io/projected/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-api-access-b7j7n" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498242    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65e4e478-1f58-498d-b6eb-619307c2943f" volumeName="kubernetes.io/empty-dir/65e4e478-1f58-498d-b6eb-619307c2943f-etc-tuned" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498254    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="694e1b6d-1615-4134-b262-e9df58af99b5" volumeName="kubernetes.io/projected/694e1b6d-1615-4134-b262-e9df58af99b5-kube-api-access-s87l5" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498295    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ccdcc49-f072-4e15-b021-9bffc3fb674a" volumeName="kubernetes.io/projected/7ccdcc49-f072-4e15-b021-9bffc3fb674a-kube-api-access-spbdb" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498311    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac8c7ff3-c156-4201-a297-ea6d1fd48e55" volumeName="kubernetes.io/projected/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-kube-api-access-zm2wv" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498323    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="260390fb-c26d-4989-9e8a-a104c398e1c6" volumeName="kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-image-registry-private-configuration" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498335    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="609f4320-8b1b-459e-9e9b-9aeeed31dbe2" volumeName="kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498348    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d78ab9e2-f553-441e-9f3d-6d8985085c89" volumeName="kubernetes.io/projected/d78ab9e2-f553-441e-9f3d-6d8985085c89-kube-api-access-62xwv" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498360    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cc09f0c-627f-4445-b172-9556cab2bbdd" volumeName="kubernetes.io/configmap/7cc09f0c-627f-4445-b172-9556cab2bbdd-config" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498371    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf18ddb4-723c-4954-8327-37f38b41eff1" volumeName="kubernetes.io/projected/bf18ddb4-723c-4954-8327-37f38b41eff1-kube-api-access-j45vj" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498384    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" volumeName="kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498395    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac8c7ff3-c156-4201-a297-ea6d1fd48e55" volumeName="kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovnkube-script-lib" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498407    4926 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="05c25677-4b67-4a79-bbca-25bcc3ef43f1" volumeName="kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-bound-sa-token" seLinuxMountContext=""
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498418    4926 reconstruct.go:97] "Volume reconstruction finished"
Oct 31 03:28:23.498418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.498425    4926 reconciler.go:29] "Reconciler: start to sync state"
Oct 31 03:28:23.501817 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.501796    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/1.log"
Oct 31 03:28:23.503556 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.503534    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/1.log"
Oct 31 03:28:23.527386 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.527356    4926 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 31 03:28:23.530764 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.530741    4926 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 31 03:28:23.530764 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.530764    4926 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 31 03:28:23.530947 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.530786    4926 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 31 03:28:23.530947 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:23.530825    4926 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 31 03:28:23.545586 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.545563    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/1.log"
Oct 31 03:28:23.545700 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.545597    4926 generic.go:356] "Generic (PLEG): container finished" podID="bb5340b5-c677-40d1-a448-498561c62a17" containerID="1b470c024d032c0301032eaa971e8b3d98823c8fa645a9638b1a6a876ccea7ea" exitCode=255
Oct 31 03:28:23.549140 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.549119    4926 generic.go:356] "Generic (PLEG): container finished" podID="af6bcf6162f2d61de6ecd5c68ef93ef6" containerID="fc0d11997f1713069298ecfd4af1bd8e96fd4bf4b05439a75dc21e15599b3308" exitCode=0
Oct 31 03:28:23.568971 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.568935    4926 generic.go:356] "Generic (PLEG): container finished" podID="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" containerID="ddf56ba6766402bbff126a852c27b70d3df7421db2b5300a3f3918060b1b6688" exitCode=0
Oct 31 03:28:23.568971 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.568957    4926 generic.go:356] "Generic (PLEG): container finished" podID="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" containerID="75ac26d85528a2fb4e37d0ff3c94c309ee97baf25914e946539f00f9a73939d3" exitCode=0
Oct 31 03:28:23.568971 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.568966    4926 generic.go:356] "Generic (PLEG): container finished" podID="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" containerID="d9274d110534008c7b6e283ba2c765030d67a7ddaf8306717390282dc3a00f32" exitCode=0
Oct 31 03:28:23.568971 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.568974    4926 generic.go:356] "Generic (PLEG): container finished" podID="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" containerID="2c920b7d458db552f7de0b86fecae361a303f6e866505f6f944e96800397d71d" exitCode=0
Oct 31 03:28:23.569170 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.568981    4926 generic.go:356] "Generic (PLEG): container finished" podID="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" containerID="c60712b30d2169117b282d5b021ae4918229007a0fb4ee038d9ae0bc56b6d5df" exitCode=0
Oct 31 03:28:23.569170 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.568989    4926 generic.go:356] "Generic (PLEG): container finished" podID="9193a1f4-f4aa-4364-ae16-5520e88d8cc3" containerID="49575c2e6f379d52c7800996bd05d47610cfb0d4e418f06bda2e7c535e12c360" exitCode=0
Oct 31 03:28:23.587617 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.587599    4926 manager.go:324] Recovery completed
Oct 31 03:28:23.595149 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:23.595129    4926 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-19.us-west-2.compute.internal\" not found"
Oct 31 03:28:23.604310 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.604294    4926 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:28:23.605546 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.605527    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:28:23.605649 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.605558    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:28:23.605649 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.605568    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:28:23.608550 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.608529    4926 cpu_manager.go:226] "Starting" policy="none"
Oct 31 03:28:23.608550 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.608549    4926 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 31 03:28:23.608675 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.608569    4926 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 31 03:28:23.608755 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.608738    4926 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 31 03:28:23.608813 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.608758    4926 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 31 03:28:23.608813 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.608802    4926 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 31 03:28:23.608813 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.608810    4926 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 31 03:28:23.608933 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.608817    4926 policy_none.go:50] "Start"
Oct 31 03:28:23.608933 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.608837    4926 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 31 03:28:23.608933 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.608850    4926 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 31 03:28:23.609071 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.609057    4926 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 31 03:28:23.609119 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.609079    4926 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 31 03:28:23.609119 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.609088    4926 policy_none.go:44] "Start"
Oct 31 03:28:23.609773 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.609737    4926 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:28:23.610844 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.610824    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:28:23.610921 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.610864    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:28:23.610921 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.610901    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:28:23.613886 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.613863    4926 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 31 03:28:23.618478 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.618464    4926 manager.go:341] "Starting Device Plugin manager"
Oct 31 03:28:23.618563 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:23.618492    4926 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 31 03:28:23.618563 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.618506    4926 server.go:87] "Starting device plugin registration server"
Oct 31 03:28:23.618864 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.618842    4926 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 31 03:28:23.618947 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.618858    4926 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 31 03:28:23.619001 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.618967    4926 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 31 03:28:23.619089 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.619056    4926 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 31 03:28:23.619143 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.619099    4926 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 31 03:28:23.619143 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.619107    4926 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 31 03:28:23.619235 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.619157    4926 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-31T03:28:23.619088302Z","UUID":"dfac3f2e-41c1-44d6-9426-5eddd72f02a2","Handler":null,"Name":"","Endpoint":""}
Oct 31 03:28:23.620549 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.620531    4926 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 31 03:28:23.620650 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.620558    4926 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 31 03:28:23.620650 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:23.620606    4926 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 31 03:28:23.620766 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:23.620649    4926 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-19.us-west-2.compute.internal\" not found"
Oct 31 03:28:23.631016 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.630987    4926 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal"]
Oct 31 03:28:23.631103 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.631093    4926 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:28:23.631848 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.631834    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:28:23.631909 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.631863    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:28:23.631909 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.631876    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:28:23.632159 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.632149    4926 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:28:23.632312 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.632300    4926 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:28:23.632913 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.632897    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:28:23.632963 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.632924    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:28:23.632963 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.632934    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:28:23.633035 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.633003    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:28:23.633035 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.633029    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:28:23.633098 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.633042    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:28:23.633187 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:23.633176    4926 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-19.us-west-2.compute.internal\" not found" node="ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.633233 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.633204    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal" event={"ID":"af6bcf6162f2d61de6ecd5c68ef93ef6","Type":"ContainerStarted","Data":"b5313ea118942dc9041ea29df0cae19b6d4e96492a2929656cde4ad78257a194"}
Oct 31 03:28:23.633276 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.633243    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal" event={"ID":"af6bcf6162f2d61de6ecd5c68ef93ef6","Type":"ContainerDied","Data":"fc0d11997f1713069298ecfd4af1bd8e96fd4bf4b05439a75dc21e15599b3308"}
Oct 31 03:28:23.633276 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.633257    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal" event={"ID":"af6bcf6162f2d61de6ecd5c68ef93ef6","Type":"ContainerStarted","Data":"e820823f0827d4209635673a4e355c31477a5ed86dc6d50ad571e6ce46a7f151"}
Oct 31 03:28:23.633345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.633297    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal" event={"ID":"8cce19b2c56183ccd97ae9b3e0b6ff57","Type":"ContainerStarted","Data":"d45a9f2c7082837e50c5763363a27bd8114f754e6cf347c4ff877d435482743b"}
Oct 31 03:28:23.633345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.633301    4926 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:28:23.633345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.633310    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal" event={"ID":"8cce19b2c56183ccd97ae9b3e0b6ff57","Type":"ContainerStarted","Data":"e096f86fb472ca5958f5b1079e2a53a7423eedc1a37fdc1c2df72f9989337a96"}
Oct 31 03:28:23.634100 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.634088    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:28:23.634155 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.634108    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:28:23.634155 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.634118    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:28:23.634227 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:23.634219    4926 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-19.us-west-2.compute.internal\" not found" node="ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.676514 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.676491    4926 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 31 03:28:23.677414 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.677398    4926 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:28:23.678833 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.678820    4926 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:28:23.719388 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.719365    4926 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 31 03:28:23.720299 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.720285    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 31 03:28:23.720368 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.720316    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 31 03:28:23.720368 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.720329    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 31 03:28:23.720428 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.720388    4926 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.724865 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.724843    4926 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:28:23.726599 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.726578    4926 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 31 03:28:23.777087 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.777061    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af6bcf6162f2d61de6ecd5c68ef93ef6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"af6bcf6162f2d61de6ecd5c68ef93ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.777185 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.777092    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8cce19b2c56183ccd97ae9b3e0b6ff57-config\") pod \"kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"8cce19b2c56183ccd97ae9b3e0b6ff57\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.777185 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.777109    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/af6bcf6162f2d61de6ecd5c68ef93ef6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"af6bcf6162f2d61de6ecd5c68ef93ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.796193 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.796174    4926 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.825645 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.825625    4926 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.825716 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.825681    4926 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.826177 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.826163    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeNotReady"
Oct 31 03:28:23.826234 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.826192    4926 setters.go:546] "Node became not ready" node="ip-10-0-130-19.us-west-2.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-10-31T03:28:23Z","lastTransitionTime":"2026-10-31T03:28:23Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Oct 31 03:28:23.834281 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.834247    4926 nodeinfomanager.go:471] existing CSINode "ip-10-0-130-19.us-west-2.compute.internal" is owned by different node (oldNodeID="c7ec8608-0ac9-4d0d-80f4-be6b09ac1792", newNodeID=""), cleaning up...
Oct 31 03:28:23.854241 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.854225    4926 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 03:28:23.854321 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:23.854260    4926 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.854321 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.854286    4926 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.877282 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.877249    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/af6bcf6162f2d61de6ecd5c68ef93ef6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"af6bcf6162f2d61de6ecd5c68ef93ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.877355 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.877287    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af6bcf6162f2d61de6ecd5c68ef93ef6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"af6bcf6162f2d61de6ecd5c68ef93ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.877355 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.877307    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8cce19b2c56183ccd97ae9b3e0b6ff57-config\") pod \"kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"8cce19b2c56183ccd97ae9b3e0b6ff57\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.877355 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.877349    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/af6bcf6162f2d61de6ecd5c68ef93ef6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"af6bcf6162f2d61de6ecd5c68ef93ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.877461 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.877352    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/af6bcf6162f2d61de6ecd5c68ef93ef6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"af6bcf6162f2d61de6ecd5c68ef93ef6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.877461 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.877393    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8cce19b2c56183ccd97ae9b3e0b6ff57-config\") pod \"kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal\" (UID: \"8cce19b2c56183ccd97ae9b3e0b6ff57\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:23.913979 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:23.913953    4926 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 03:28:23.914067 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:23.913996    4926 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:24.095419 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.095351    4926 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-19.us-west-2.compute.internal" event="NodeReady"
Oct 31 03:28:24.095554 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.095433    4926 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 31 03:28:24.474327 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.474299    4926 apiserver.go:52] "Watching apiserver"
Oct 31 03:28:24.575756 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.575726    4926 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 31 03:28:24.576375 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.576349    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-zwpfx","kube-system/kas-connection-checker-685895c7c9-kfthp","openshift-image-registry/node-ca-fd7k8","openshift-insights/insights-operator-7c4986d449-s42wk","openshift-ovn-kubernetes/ovnkube-node-wwpg9","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg","openshift-image-registry/image-registry-5cb458f74f-7bjpx","kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal","openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2","openshift-ingress-canary/ingress-canary-pq8vh","openshift-network-diagnostics/network-check-target-n6jzp","kube-system/kas-connection-checker-685895c7c9-8pxph","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-d6qdw","kube-system/kas-connection-checker-685895c7c9-htdq6","openshift-multus/multus-additional-cni-plugins-xtmjj","openshift-kube-storage-version-migrator/migrator-66df5d496b-mft5z","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4","openshift-dns/dns-default-zcpmp","openshift-dns/node-resolver-g95zb","openshift-insights/insights-runtime-extractor-tw5k4","kube-system/global-pull-secret-syncer-phb2z","openshift-console-operator/console-operator-7cd56b8dbc-wpxfm","openshift-network-console/networking-console-plugin-86c478bb88-k2fcf","kube-system/konnectivity-agent-qwh92","openshift-network-diagnostics/network-check-source-6f78d59f99-5jgbv","openshift-multus/network-metrics-daemon-65ph6","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr","openshift-cluster-node-tuning-operator/tuned-z9phd","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal","openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n","openshift-ingress/router-default-77685b76bf-cmqhp","openshift-multus/multus-c2qwl","openshift-image-registry/image-registry-8bd6c7ddd-l84xn"]
Oct 31 03:28:24.577630 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.577615    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:24.578613 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.578586    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:24.579356 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.579333    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:24.580907 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.580890    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.581328 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.581305    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:24.582927 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.582911    4926 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:24.583206 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.583190    4926 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:24.586524 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.586503    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:24.590536 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.590507    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.592261 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.592221    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:24.592780 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.592765    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:24.596703 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.596684    4926 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 31 03:28:24.626666 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.626643    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.626840 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.626825    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.627434 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.627419    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 31 03:28:24.627434 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.627423    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 31 03:28:24.627519 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.627482    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 31 03:28:24.628179 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.628163    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 31 03:28:24.628211 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.628201    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 31 03:28:24.628211 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.628206    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 31 03:28:24.628607 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.628592    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 31 03:28:24.628676 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.628630    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 31 03:28:24.628944 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.628930    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 31 03:28:24.629221 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.629177    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.629544 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.629331    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.629544 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.629374    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.629544 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.629386    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.629544 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.629419    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.629544 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.629438    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 31 03:28:24.629544 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.629454    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 31 03:28:24.629544 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.629460    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.630195 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.629553    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 31 03:28:24.630195 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.629613    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.630195 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.629692    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.630195 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.629795    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 31 03:28:24.630195 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.629794    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.630195 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.629882    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.630195 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.629930    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 31 03:28:24.676320 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.676294    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 31 03:28:24.676469 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.676453    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 31 03:28:24.682223 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682206    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-multus-certs\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.682287 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682235    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a-cert\") pod \"ingress-canary-pq8vh\" (UID: \"fc6215a3-f0b6-4f89-9a0d-30589bb43c5a\") " pod="openshift-ingress-canary/ingress-canary-pq8vh"
Oct 31 03:28:24.682287 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682254    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-device-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.682357 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682286    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/694e1b6d-1615-4134-b262-e9df58af99b5-tmp-dir\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:28:24.682357 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682339    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r69c\" (UniqueName: \"kubernetes.io/projected/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-kube-api-access-4r69c\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:24.682456 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682430    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cnibin\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.682456 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682442    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/694e1b6d-1615-4134-b262-e9df58af99b5-tmp-dir\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:28:24.682568 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682450    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-certificates\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.682568 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682491    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sklx7\" (UniqueName: \"kubernetes.io/projected/8d7cafa8-5ce8-4991-8598-5e03ebfb0b75-kube-api-access-sklx7\") pod \"network-check-source-6f78d59f99-5jgbv\" (UID: \"8d7cafa8-5ce8-4991-8598-5e03ebfb0b75\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-5jgbv"
Oct 31 03:28:24.682568 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682519    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ztgb\" (UniqueName: \"kubernetes.io/projected/65e4e478-1f58-498d-b6eb-619307c2943f-kube-api-access-6ztgb\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.682568 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682552    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62xwv\" (UniqueName: \"kubernetes.io/projected/d78ab9e2-f553-441e-9f3d-6d8985085c89-kube-api-access-62xwv\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:24.682759 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682577    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-image-registry-private-configuration\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.682759 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682605    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-daemon-config\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.682759 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682633    4926 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 03:28:24.682759 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.682679    4926 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:24.682759 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682695    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df73b930-8122-434b-b2a0-cc3dd2c44d5c-tmp\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:24.682953 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682629    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df73b930-8122-434b-b2a0-cc3dd2c44d5c-tmp\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:24.682953 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682811    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-trusted-ca\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.682953 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682834    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovnkube-config\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.682953 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682857    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovnkube-script-lib\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.682953 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682885    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-modprobe-d\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.682953 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682908    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7j7n\" (UniqueName: \"kubernetes.io/projected/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-api-access-b7j7n\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:24.683242 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.682949    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-image-registry-private-configuration\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.683242 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683083    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdrgg\" (UniqueName: \"kubernetes.io/projected/02dec91e-45a1-4609-9b09-72ba5259bd0e-kube-api-access-bdrgg\") pod \"kas-connection-checker-685895c7c9-kfthp\" (UID: \"02dec91e-45a1-4609-9b09-72ba5259bd0e\") " pod="kube-system/kas-connection-checker-685895c7c9-kfthp"
Oct 31 03:28:24.683242 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683116    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbg5x\" (UniqueName: \"kubernetes.io/projected/cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a-kube-api-access-pbg5x\") pod \"kas-connection-checker-685895c7c9-htdq6\" (UID: \"cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a\") " pod="kube-system/kas-connection-checker-685895c7c9-htdq6"
Oct 31 03:28:24.683242 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683151    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysconfig\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.683242 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683172    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a52638f4-7a5b-45de-9054-d173d66c7521-tmp-dir\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:24.683242 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683189    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/694e1b6d-1615-4134-b262-e9df58af99b5-hosts-file\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:28:24.683242 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683205    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.683242 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683230    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.683598 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683254    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-nginx-conf\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:24.683598 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683323    4926 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 31 03:28:24.683598 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683326    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a52638f4-7a5b-45de-9054-d173d66c7521-tmp-dir\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:24.683598 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683337    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-kubernetes\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.683598 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.683361    4926 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-19.us-west-2.compute.internal"
Oct 31 03:28:24.683598 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683395    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5615d56d-7e1b-4f88-803c-b6a1c0435263-data-volume\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:24.683598 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683423    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-socket-dir-parent\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.683598 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683445    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-cni-bin\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.683598 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683493    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysctl-conf\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.683598 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683511    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5615d56d-7e1b-4f88-803c-b6a1c0435263-data-volume\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:24.683598 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683522    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxbxj\" (UniqueName: \"kubernetes.io/projected/aef16625-d836-42cb-bf49-176f80e73474-kube-api-access-gxbxj\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:24.683598 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683589    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-lib-modules\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.683913 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683612    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:24.683913 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683728    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.683913 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683764    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dfdt\" (UniqueName: \"kubernetes.io/projected/df73b930-8122-434b-b2a0-cc3dd2c44d5c-kube-api-access-7dfdt\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:24.683913 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683801    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-sys\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.683913 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683832    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a52638f4-7a5b-45de-9054-d173d66c7521-config-volume\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:24.684043 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683930    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f03b006-c2d0-4e6b-920b-88d146280084-serviceca\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:28:24.684043 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.683968    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.684043 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684010    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-var-lib-kubelet\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.684043 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684030    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-certificates\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.684153 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684059    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-etc-kubernetes\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.684153 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684104    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spbdb\" (UniqueName: \"kubernetes.io/projected/7ccdcc49-f072-4e15-b021-9bffc3fb674a-kube-api-access-spbdb\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.684153 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684136    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-etc-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.684242 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684159    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-default-certificate\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:24.684242 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684175    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:24.684242 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684198    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:24.684380 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684285    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-installation-pull-secrets\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.684380 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684333    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drh58\" (UniqueName: \"kubernetes.io/projected/0d9f6de8-463e-4779-9825-e4318b9273b4-kube-api-access-drh58\") pod \"volume-data-source-validator-58c9f5f46d-d6qdw\" (UID: \"0d9f6de8-463e-4779-9825-e4318b9273b4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-d6qdw"
Oct 31 03:28:24.684470 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684413    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kk5j\" (UniqueName: \"kubernetes.io/projected/0bf0c577-0537-46fd-950a-ce5f565a51da-kube-api-access-6kk5j\") pod \"service-ca-operator-544d749c8c-nhj2n\" (UID: \"0bf0c577-0537-46fd-950a-ce5f565a51da\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:24.684470 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684418    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-default-certificate\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:24.684524 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684515    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6q9r\" (UniqueName: \"kubernetes.io/projected/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a-kube-api-access-c6q9r\") pod \"ingress-canary-pq8vh\" (UID: \"fc6215a3-f0b6-4f89-9a0d-30589bb43c5a\") " pod="openshift-ingress-canary/ingress-canary-pq8vh"
Oct 31 03:28:24.684556 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684543    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-run-netns\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.684591 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684560    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-cni-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.684591 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684578    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05c25677-4b67-4a79-bbca-25bcc3ef43f1-ca-trust-extracted\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.684662 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684613    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-slash\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.684698 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684672    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05c25677-4b67-4a79-bbca-25bcc3ef43f1-ca-trust-extracted\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.684735 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684699    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7svn9\" (UniqueName: \"kubernetes.io/projected/4700dca5-f15a-4329-b719-828f5475810e-kube-api-access-7svn9\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:28:24.684735 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684720    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgprc\" (UniqueName: \"kubernetes.io/projected/3add72f4-7034-4186-95b5-59d2bd770827-kube-api-access-tgprc\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.684824 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684736    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65e4e478-1f58-498d-b6eb-619307c2943f-tmp\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.684824 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684751    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/260390fb-c26d-4989-9e8a-a104c398e1c6-ca-trust-extracted\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.684824 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684766    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-dbus\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:24.684824 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684784    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-549vb\" (UniqueName: \"kubernetes.io/projected/bb5340b5-c677-40d1-a448-498561c62a17-kube-api-access-549vb\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:24.685014 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684824    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-stats-auth\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:24.685014 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684865    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/260390fb-c26d-4989-9e8a-a104c398e1c6-ca-trust-extracted\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.685014 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684864    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-k8s-cni-cncf-io\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.685014 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684970    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65e4e478-1f58-498d-b6eb-619307c2943f-tmp\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.685014 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.684993    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:24.685193 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685017    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0bf0c577-0537-46fd-950a-ce5f565a51da-config\") pod \"service-ca-operator-544d749c8c-nhj2n\" (UID: \"0bf0c577-0537-46fd-950a-ce5f565a51da\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:24.685193 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685044    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfc5g\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-kube-api-access-pfc5g\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.685193 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685076    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-systemd\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.685193 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685088    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-stats-auth\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:24.685193 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685108    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-host\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.685193 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685152    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s87l5\" (UniqueName: \"kubernetes.io/projected/694e1b6d-1615-4134-b262-e9df58af99b5-kube-api-access-s87l5\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:28:24.685193 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685178    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5js2v\" (UniqueName: \"kubernetes.io/projected/7cc09f0c-627f-4445-b172-9556cab2bbdd-kube-api-access-5js2v\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg\" (UID: \"7cc09f0c-627f-4445-b172-9556cab2bbdd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:24.685193 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685195    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0bf0c577-0537-46fd-950a-ce5f565a51da-serving-cert\") pod \"service-ca-operator-544d749c8c-nhj2n\" (UID: \"0bf0c577-0537-46fd-950a-ce5f565a51da\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:24.685523 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685204    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0bf0c577-0537-46fd-950a-ce5f565a51da-config\") pod \"service-ca-operator-544d749c8c-nhj2n\" (UID: \"0bf0c577-0537-46fd-950a-ce5f565a51da\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:24.685523 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685261    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-cni-multus\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.685523 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685303    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:24.685523 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685331    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scqrb\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-kube-api-access-scqrb\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.685523 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685382    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0bf0c577-0537-46fd-950a-ce5f565a51da-serving-cert\") pod \"service-ca-operator-544d749c8c-nhj2n\" (UID: \"0bf0c577-0537-46fd-950a-ce5f565a51da\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:24.685523 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685367    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0883de4-ab87-4a25-bd8b-ecd4914d4638-agent-certs\") pod \"konnectivity-agent-qwh92\" (UID: \"a0883de4-ab87-4a25-bd8b-ecd4914d4638\") " pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:28:24.685523 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685433    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cc09f0c-627f-4445-b172-9556cab2bbdd-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg\" (UID: \"7cc09f0c-627f-4445-b172-9556cab2bbdd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:24.685744 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685526    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:24.685744 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685537    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm2wv\" (UniqueName: \"kubernetes.io/projected/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-kube-api-access-zm2wv\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.685744 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685602    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:24.685744 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685637    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jdm6\" (UniqueName: \"kubernetes.io/projected/5f03b006-c2d0-4e6b-920b-88d146280084-kube-api-access-7jdm6\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:28:24.685744 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685708    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-trusted-ca\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.685939 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685789    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-cnibin\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.685939 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685820    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-conf-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.685939 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685880    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-kubelet-config\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:24.685939 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.685907    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j45vj\" (UniqueName: \"kubernetes.io/projected/bf18ddb4-723c-4954-8327-37f38b41eff1-kube-api-access-j45vj\") pod \"kas-connection-checker-685895c7c9-8pxph\" (UID: \"bf18ddb4-723c-4954-8327-37f38b41eff1\") " pod="kube-system/kas-connection-checker-685895c7c9-8pxph"
Oct 31 03:28:24.686062 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686009    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-env-overrides\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.686062 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686028    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56fwn\" (UniqueName: \"kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn\") pod \"network-check-target-n6jzp\" (UID: \"05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8\") " pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:28:24.686062 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686052    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-run-ovn-kubernetes\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.686157 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686067    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysctl-d\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.686227 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686215    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-socket-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.686258 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686238    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bb5340b5-c677-40d1-a448-498561c62a17-config\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:24.686305 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686256    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-installation-pull-secrets\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.686334 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686306    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4700dca5-f15a-4329-b719-828f5475810e-host-slash\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:28:24.686334 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686329    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0883de4-ab87-4a25-bd8b-ecd4914d4638-konnectivity-ca\") pod \"konnectivity-agent-qwh92\" (UID: \"a0883de4-ab87-4a25-bd8b-ecd4914d4638\") " pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:28:24.686420 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686345    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-netns\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.686420 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686363    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfb57\" (UniqueName: \"kubernetes.io/projected/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-kube-api-access-rfb57\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.686420 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686400    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df73b930-8122-434b-b2a0-cc3dd2c44d5c-serving-cert\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:24.686566 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686411    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bb5340b5-c677-40d1-a448-498561c62a17-config\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:24.686566 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686502    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bb5340b5-c677-40d1-a448-498561c62a17-trusted-ca\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:24.686566 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686539    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df73b930-8122-434b-b2a0-cc3dd2c44d5c-serving-cert\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:24.686566 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686544    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:24.686695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686577    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-registration-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.686695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686595    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:24.686695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686620    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.686695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686636    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-kubelet\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.686695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686650    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-systemd\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.686695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686666    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-log-socket\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.686695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686680    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-system-cni-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.686941 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686719    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.686941 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686744    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bb5340b5-c677-40d1-a448-498561c62a17-trusted-ca\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:24.686941 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686771    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/df73b930-8122-434b-b2a0-cc3dd2c44d5c-snapshots\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:24.686941 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686795    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-var-lib-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.686941 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686844    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/df73b930-8122-434b-b2a0-cc3dd2c44d5c-snapshots\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:24.686941 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686876    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-etc-selinux\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.686941 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686909    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7cc09f0c-627f-4445-b172-9556cab2bbdd-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg\" (UID: \"7cc09f0c-627f-4445-b172-9556cab2bbdd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:24.686941 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.686938    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df73b930-8122-434b-b2a0-cc3dd2c44d5c-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:24.687278 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687004    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bb5340b5-c677-40d1-a448-498561c62a17-serving-cert\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:24.687278 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687035    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-node-log\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.687278 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687063    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4700dca5-f15a-4329-b719-828f5475810e-iptables-alerter-script\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:28:24.687278 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687091    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7wqg\" (UniqueName: \"kubernetes.io/projected/945a916b-41c5-46f6-9c1a-9a8c93f7d6d2-kube-api-access-h7wqg\") pod \"migrator-66df5d496b-mft5z\" (UID: \"945a916b-41c5-46f6-9c1a-9a8c93f7d6d2\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mft5z"
Oct 31 03:28:24.687278 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687134    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l8rz\" (UniqueName: \"kubernetes.io/projected/26a79eda-be35-4aa8-b10b-cb2b1133a086-kube-api-access-2l8rz\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:24.687278 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687219    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-hostroot\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.687278 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687257    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df73b930-8122-434b-b2a0-cc3dd2c44d5c-service-ca-bundle\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:24.687563 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687328    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.687563 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687342    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bb5340b5-c677-40d1-a448-498561c62a17-serving-cert\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:24.687563 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687403    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcq98\" (UniqueName: \"kubernetes.io/projected/a52638f4-7a5b-45de-9054-d173d66c7521-kube-api-access-gcq98\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:24.687563 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687469    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df73b930-8122-434b-b2a0-cc3dd2c44d5c-service-ca-bundle\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:24.687563 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687478    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-systemd-units\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.687563 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687523    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/65e4e478-1f58-498d-b6eb-619307c2943f-etc-tuned\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.687563 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687548    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5615d56d-7e1b-4f88-803c-b6a1c0435263-crio-socket\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:24.687868 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687572    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-cni-bin\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.687868 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687613    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-kubelet\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.687868 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687635    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/65e4e478-1f58-498d-b6eb-619307c2943f-etc-tuned\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.687868 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687650    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-system-cni-dir\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.687868 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687703    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-bound-sa-token\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.687868 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687739    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovn-node-metrics-cert\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.687868 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687748    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df73b930-8122-434b-b2a0-cc3dd2c44d5c-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:24.687868 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687801    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-run\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.688137 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687872    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f03b006-c2d0-4e6b-920b-88d146280084-host\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:28:24.688137 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687908    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:24.688137 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687931    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.688137 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687951    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-sys-fs\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.688137 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687970    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:24.688137 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.687988    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ccdcc49-f072-4e15-b021-9bffc3fb674a-cni-binary-copy\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.688137 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.688004    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aef16625-d836-42cb-bf49-176f80e73474-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:24.688137 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.688027    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:24.688137 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.688094    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.688137 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.688126    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-ovn\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.688461 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.688153    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-cni-netd\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.688461 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.688186    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-bound-sa-token\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.688461 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.688282    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aef16625-d836-42cb-bf49-176f80e73474-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:24.728838 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.728774    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.729293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729220    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 31 03:28:24.729293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729245    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.729293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729288    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.729642 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729317    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 31 03:28:24.729642 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729331    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.729642 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729472    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 31 03:28:24.729642 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729490    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 31 03:28:24.729642 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729523    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.729642 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729223    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729686    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729741    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729742    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729775    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729799    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729891    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729897    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729910    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729919    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.729937    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.730028    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.730047    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.730054    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.730083    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.730104    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.730112    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.730161    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.730260    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 31 03:28:24.730508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.730294    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.732234 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.730584    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.732234 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.730643    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 31 03:28:24.732234 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.730662    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 31 03:28:24.732234 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.730681    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.732234 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.731126    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 31 03:28:24.732234 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.731298    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-8nlcv\""
Oct 31 03:28:24.732234 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.731416    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.732234 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.731634    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-q48mz\""
Oct 31 03:28:24.732234 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.732121    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-dvnpq\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.733200    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hp8t5\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.733289    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jdz67\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.733340    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.733542    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.733671    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zvcgd\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.733710    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.733757    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.733784    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.733743    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-nginx-conf\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.733848    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7b9lc\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.733879    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.733910    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-dlcmg\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.733946    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.734062    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.734161    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.734212    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.734251    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.734307    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.734366    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.734430    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.734646    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z9sxx\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.734809    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hztlx\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.735007    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-installation-pull-secrets\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.735216    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.735258    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.735341    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-trusted-ca\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.735392    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2rc66\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.735487    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.735519    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-image-registry-private-configuration\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.735538    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vvwkt\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.735601    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-8f2x4\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.735695    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.735750    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a52638f4-7a5b-45de-9054-d173d66c7521-config-volume\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:24.735816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.735749    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 31 03:28:24.736915 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.735807    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovnkube-script-lib\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.736915 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.735945    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-lj989\""
Oct 31 03:28:24.736915 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.735941    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hp5ds\""
Oct 31 03:28:24.736915 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.736014    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a0883de4-ab87-4a25-bd8b-ecd4914d4638-agent-certs\") pod \"konnectivity-agent-qwh92\" (UID: \"a0883de4-ab87-4a25-bd8b-ecd4914d4638\") " pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:28:24.736915 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.736047    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xg7xf\""
Oct 31 03:28:24.736915 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.736048    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovnkube-config\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.736915 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.736128    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7cc09f0c-627f-4445-b172-9556cab2bbdd-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg\" (UID: \"7cc09f0c-627f-4445-b172-9556cab2bbdd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:24.736915 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.736151    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-trusted-ca\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.736915 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.736134    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fs258\""
Oct 31 03:28:24.736915 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.736195    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 31 03:28:24.736915 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.736348    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-daemon-config\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.736915 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.736362    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-env-overrides\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.736915 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.736669    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a0883de4-ab87-4a25-bd8b-ecd4914d4638-konnectivity-ca\") pod \"konnectivity-agent-qwh92\" (UID: \"a0883de4-ab87-4a25-bd8b-ecd4914d4638\") " pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:28:24.736915 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.736712    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-installation-pull-secrets\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.736915 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.736806    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cni-binary-copy\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.737380 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.736944    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.737380 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.737124    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7cc09f0c-627f-4445-b172-9556cab2bbdd-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg\" (UID: \"7cc09f0c-627f-4445-b172-9556cab2bbdd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:24.737836 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.737819    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4700dca5-f15a-4329-b719-828f5475810e-iptables-alerter-script\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:28:24.737929 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.737916    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-ovn-node-metrics-cert\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.738343 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.738324    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7ccdcc49-f072-4e15-b021-9bffc3fb674a-cni-binary-copy\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.738405 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.738370    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-original-pull-secret\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:24.739751 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.739732    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62xwv\" (UniqueName: \"kubernetes.io/projected/d78ab9e2-f553-441e-9f3d-6d8985085c89-kube-api-access-62xwv\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:24.739902 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.739880    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sklx7\" (UniqueName: \"kubernetes.io/projected/8d7cafa8-5ce8-4991-8598-5e03ebfb0b75-kube-api-access-sklx7\") pod \"network-check-source-6f78d59f99-5jgbv\" (UID: \"8d7cafa8-5ce8-4991-8598-5e03ebfb0b75\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-5jgbv"
Oct 31 03:28:24.740013 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.739999    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ztgb\" (UniqueName: \"kubernetes.io/projected/65e4e478-1f58-498d-b6eb-619307c2943f-kube-api-access-6ztgb\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.740044 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.740021    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7j7n\" (UniqueName: \"kubernetes.io/projected/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-api-access-b7j7n\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:24.740084 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.740063    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r69c\" (UniqueName: \"kubernetes.io/projected/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-kube-api-access-4r69c\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:24.740744 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.740726    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbg5x\" (UniqueName: \"kubernetes.io/projected/cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a-kube-api-access-pbg5x\") pod \"kas-connection-checker-685895c7c9-htdq6\" (UID: \"cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a\") " pod="kube-system/kas-connection-checker-685895c7c9-htdq6"
Oct 31 03:28:24.740924 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.740909    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dfdt\" (UniqueName: \"kubernetes.io/projected/df73b930-8122-434b-b2a0-cc3dd2c44d5c-kube-api-access-7dfdt\") pod \"insights-operator-7c4986d449-s42wk\" (UID: \"df73b930-8122-434b-b2a0-cc3dd2c44d5c\") " pod="openshift-insights/insights-operator-7c4986d449-s42wk"
Oct 31 03:28:24.741425 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.741408    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drh58\" (UniqueName: \"kubernetes.io/projected/0d9f6de8-463e-4779-9825-e4318b9273b4-kube-api-access-drh58\") pod \"volume-data-source-validator-58c9f5f46d-d6qdw\" (UID: \"0d9f6de8-463e-4779-9825-e4318b9273b4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-d6qdw"
Oct 31 03:28:24.742898 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.742882    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-image-registry-private-configuration\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.742981 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.742966    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a-cert\") pod \"ingress-canary-pq8vh\" (UID: \"fc6215a3-f0b6-4f89-9a0d-30589bb43c5a\") " pod="openshift-ingress-canary/ingress-canary-pq8vh"
Oct 31 03:28:24.743032 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.743010    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-certificates\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.743364 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.743350    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxbxj\" (UniqueName: \"kubernetes.io/projected/aef16625-d836-42cb-bf49-176f80e73474-kube-api-access-gxbxj\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:24.744453 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.744439    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-certificates\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.744528 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.744507    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5f03b006-c2d0-4e6b-920b-88d146280084-serviceca\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:28:24.744888 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.744869    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spbdb\" (UniqueName: \"kubernetes.io/projected/7ccdcc49-f072-4e15-b021-9bffc3fb674a-kube-api-access-spbdb\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.744920 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.744871    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdrgg\" (UniqueName: \"kubernetes.io/projected/02dec91e-45a1-4609-9b09-72ba5259bd0e-kube-api-access-bdrgg\") pod \"kas-connection-checker-685895c7c9-kfthp\" (UID: \"02dec91e-45a1-4609-9b09-72ba5259bd0e\") " pod="kube-system/kas-connection-checker-685895c7c9-kfthp"
Oct 31 03:28:24.748828 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.748808    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgprc\" (UniqueName: \"kubernetes.io/projected/3add72f4-7034-4186-95b5-59d2bd770827-kube-api-access-tgprc\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.748927 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.748874    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j45vj\" (UniqueName: \"kubernetes.io/projected/bf18ddb4-723c-4954-8327-37f38b41eff1-kube-api-access-j45vj\") pod \"kas-connection-checker-685895c7c9-8pxph\" (UID: \"bf18ddb4-723c-4954-8327-37f38b41eff1\") " pod="kube-system/kas-connection-checker-685895c7c9-8pxph"
Oct 31 03:28:24.751420 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.751401    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jdm6\" (UniqueName: \"kubernetes.io/projected/5f03b006-c2d0-4e6b-920b-88d146280084-kube-api-access-7jdm6\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:28:24.751604 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.751587    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5js2v\" (UniqueName: \"kubernetes.io/projected/7cc09f0c-627f-4445-b172-9556cab2bbdd-kube-api-access-5js2v\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg\" (UID: \"7cc09f0c-627f-4445-b172-9556cab2bbdd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg"
Oct 31 03:28:24.751710 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.751696    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6q9r\" (UniqueName: \"kubernetes.io/projected/fc6215a3-f0b6-4f89-9a0d-30589bb43c5a-kube-api-access-c6q9r\") pod \"ingress-canary-pq8vh\" (UID: \"fc6215a3-f0b6-4f89-9a0d-30589bb43c5a\") " pod="openshift-ingress-canary/ingress-canary-pq8vh"
Oct 31 03:28:24.751773 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.751729    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-549vb\" (UniqueName: \"kubernetes.io/projected/bb5340b5-c677-40d1-a448-498561c62a17-kube-api-access-549vb\") pod \"console-operator-7cd56b8dbc-wpxfm\" (UID: \"bb5340b5-c677-40d1-a448-498561c62a17\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:24.752176 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.752156    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s87l5\" (UniqueName: \"kubernetes.io/projected/694e1b6d-1615-4134-b262-e9df58af99b5-kube-api-access-s87l5\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:28:24.753623 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.753571    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56fwn\" (UniqueName: \"kubernetes.io/projected/05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8-kube-api-access-56fwn\") pod \"network-check-target-n6jzp\" (UID: \"05d1cd22-fa2c-4acc-84e6-0dd1dad24bd8\") " pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:28:24.753796 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.753770    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scqrb\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-kube-api-access-scqrb\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.754084 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.754065    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7svn9\" (UniqueName: \"kubernetes.io/projected/4700dca5-f15a-4329-b719-828f5475810e-kube-api-access-7svn9\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:28:24.754260 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.754246    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfb57\" (UniqueName: \"kubernetes.io/projected/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-kube-api-access-rfb57\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.754361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.754249    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfc5g\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-kube-api-access-pfc5g\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.754361 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.754345    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l8rz\" (UniqueName: \"kubernetes.io/projected/26a79eda-be35-4aa8-b10b-cb2b1133a086-kube-api-access-2l8rz\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:24.754917 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.754902    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-bound-sa-token\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.755164 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.755149    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm2wv\" (UniqueName: \"kubernetes.io/projected/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-kube-api-access-zm2wv\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.755343 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.755326    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-bound-sa-token\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.755461 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.755447    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7wqg\" (UniqueName: \"kubernetes.io/projected/945a916b-41c5-46f6-9c1a-9a8c93f7d6d2-kube-api-access-h7wqg\") pod \"migrator-66df5d496b-mft5z\" (UID: \"945a916b-41c5-46f6-9c1a-9a8c93f7d6d2\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mft5z"
Oct 31 03:28:24.755735 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.755719    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kk5j\" (UniqueName: \"kubernetes.io/projected/0bf0c577-0537-46fd-950a-ce5f565a51da-kube-api-access-6kk5j\") pod \"service-ca-operator-544d749c8c-nhj2n\" (UID: \"0bf0c577-0537-46fd-950a-ce5f565a51da\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n"
Oct 31 03:28:24.755775 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.755725    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcq98\" (UniqueName: \"kubernetes.io/projected/a52638f4-7a5b-45de-9054-d173d66c7521-kube-api-access-gcq98\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:24.789059 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789034    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:24.789166 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789071    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-cnibin\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.789166 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789093    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-conf-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.789166 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789116    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-kubelet-config\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:24.789166 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789141    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-run-ovn-kubernetes\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789174    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-conf-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789163    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysctl-d\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789190    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-kubelet-config\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.789201    4926 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789176    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-cnibin\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789218    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-socket-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789232    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-run-ovn-kubernetes\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789249    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4700dca5-f15a-4329-b719-828f5475810e-host-slash\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.789301    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls podName:a52638f4-7a5b-45de-9054-d173d66c7521 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:25.289259163 +0000 UTC m=+1.874799423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls") pod "dns-default-zcpmp" (UID: "a52638f4-7a5b-45de-9054-d173d66c7521") : secret "dns-default-metrics-tls" not found
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789302    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4700dca5-f15a-4329-b719-828f5475810e-host-slash\") pod \"iptables-alerter-zwpfx\" (UID: \"4700dca5-f15a-4329-b719-828f5475810e\") " pod="openshift-network-operator/iptables-alerter-zwpfx"
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789320    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysctl-d\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789323    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-socket-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789341    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-netns\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789363    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789378    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-registration-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.789385 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789393    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-netns\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789395    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789431    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-kubelet\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.789459    4926 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789470    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-registration-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.789480    4926 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.789504    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls podName:aef16625-d836-42cb-bf49-176f80e73474 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:25.289488945 +0000 UTC m=+1.875029199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dwx2" (UID: "aef16625-d836-42cb-bf49-176f80e73474") : secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789511    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-systemd\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789514    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-kubelet\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.789578    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:25.289566302 +0000 UTC m=+1.875106555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : secret "router-metrics-certs-default" not found
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789474    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-systemd\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789608    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-log-socket\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789624    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-system-cni-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789640    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-var-lib-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789663    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-etc-selinux\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789687    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-node-log\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789703    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-hostroot\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789715    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-var-lib-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789721    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-log-socket\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789719    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789738    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-etc-selinux\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789747    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-node-log\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789701    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-system-cni-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789772    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-systemd-units\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.789778    4926 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.789790    4926 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bd6c7ddd-l84xn: secret "image-registry-tls" not found
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789792    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5615d56d-7e1b-4f88-803c-b6a1c0435263-crio-socket\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789800    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-systemd-units\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789771    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-hostroot\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789819    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-cni-bin\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.789835    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls podName:05c25677-4b67-4a79-bbca-25bcc3ef43f1 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:25.289823403 +0000 UTC m=+1.875363669 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls") pod "image-registry-8bd6c7ddd-l84xn" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1") : secret "image-registry-tls" not found
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789858    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-kubelet\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789859    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-cni-bin\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789874    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-system-cni-dir\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789891    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5615d56d-7e1b-4f88-803c-b6a1c0435263-crio-socket\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789919    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-run\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789944    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-kubelet\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789963    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f03b006-c2d0-4e6b-920b-88d146280084-host\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789975    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-system-cni-dir\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.789976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.789976    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-run\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790007    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790022    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790048    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-sys-fs\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790061    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.790068    4926 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790119    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5f03b006-c2d0-4e6b-920b-88d146280084-host\") pod \"node-ca-fd7k8\" (UID: \"5f03b006-c2d0-4e6b-920b-88d146280084\") " pod="openshift-image-registry/node-ca-fd7k8"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790136    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-sys-fs\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790153    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.790177    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm podName:5615d56d-7e1b-4f88-803c-b6a1c0435263 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:25.290166937 +0000 UTC m=+1.875707177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tw5k4" (UID: "5615d56d-7e1b-4f88-803c-b6a1c0435263") : configmap "kube-rbac-proxy" not found
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.790197    4926 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790221    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.790241    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert podName:f4b87fa6-eceb-4783-b2c5-4685e08ba9ef nodeName:}" failed. No retries permitted until 2026-10-31 03:28:25.290221056 +0000 UTC m=+1.875761331 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k2fcf" (UID: "f4b87fa6-eceb-4783-b2c5-4685e08ba9ef") : secret "networking-console-plugin-cert" not found
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790260    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790296    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-ovn\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790316    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-cni-netd\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790331    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-multus-certs\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790353    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-device-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790379    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-cni-netd\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790379    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cnibin\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790405    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-run-ovn\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790399    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-multus-certs\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790440    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-cnibin\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790453    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3add72f4-7034-4186-95b5-59d2bd770827-device-dir\") pod \"aws-ebs-csi-driver-node-j5bd4\" (UID: \"3add72f4-7034-4186-95b5-59d2bd770827\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j5bd4"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790461    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-modprobe-d\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790504    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysconfig\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790523    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-modprobe-d\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790536    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/694e1b6d-1615-4134-b262-e9df58af99b5-hosts-file\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790561    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790570    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysconfig\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790586    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-kubernetes\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790606    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/694e1b6d-1615-4134-b262-e9df58af99b5-hosts-file\") pod \"node-resolver-g95zb\" (UID: \"694e1b6d-1615-4134-b262-e9df58af99b5\") " pod="openshift-dns/node-resolver-g95zb"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790612    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-socket-dir-parent\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.790622    4926 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.790630    4926 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb458f74f-7bjpx: secret "image-registry-tls" not found
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790635    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-cni-bin\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790637    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-kubernetes\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790658    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysctl-conf\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.790666    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls podName:260390fb-c26d-4989-9e8a-a104c398e1c6 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:25.290654095 +0000 UTC m=+1.876194337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls") pod "image-registry-5cb458f74f-7bjpx" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6") : secret "image-registry-tls" not found
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790669    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-socket-dir-parent\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790695    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-lib-modules\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790693    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-cni-bin\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790715    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790745    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790746    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-sysctl-conf\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790761    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-sys\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790779    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790786    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9193a1f4-f4aa-4364-ae16-5520e88d8cc3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xtmjj\" (UID: \"9193a1f4-f4aa-4364-ae16-5520e88d8cc3\") " pod="openshift-multus/multus-additional-cni-plugins-xtmjj"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790793    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-var-lib-kubelet\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790795    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-lib-modules\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.790801    4926 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790811    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-etc-kubernetes\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790822    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-sys\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790828    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-etc-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790830    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790869    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-etc-openvswitch\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790881    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-etc-kubernetes\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790910    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790912    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-var-lib-kubelet\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790934    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.790955    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls podName:cdbd24b7-07ab-44ba-a564-0b84466b7bfa nodeName:}" failed. No retries permitted until 2026-10-31 03:28:25.290927848 +0000 UTC m=+1.876468100 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-cwnbr" (UID: "cdbd24b7-07ab-44ba-a564-0b84466b7bfa") : secret "samples-operator-tls" not found
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.790995    4926 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791027    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-run-netns\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.790992    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-run-netns\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.791043    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls podName:5615d56d-7e1b-4f88-803c-b6a1c0435263 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:25.291030137 +0000 UTC m=+1.876570416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tw5k4" (UID: "5615d56d-7e1b-4f88-803c-b6a1c0435263") : secret "insights-runtime-extractor-tls" not found
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.791059    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:25.291050589 +0000 UTC m=+1.876590835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : configmap references non-existent config key: service-ca.crt
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791073    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-cni-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791098    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-slash\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791109    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-multus-cni-dir\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791125    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-dbus\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791165    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac8c7ff3-c156-4201-a297-ea6d1fd48e55-host-slash\") pod \"ovnkube-node-wwpg9\" (UID: \"ac8c7ff3-c156-4201-a297-ea6d1fd48e55\") " pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791165    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-k8s-cni-cncf-io\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791193    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-run-k8s-cni-cncf-io\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791200    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791206    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/609f4320-8b1b-459e-9e9b-9aeeed31dbe2-dbus\") pod \"global-pull-secret-syncer-phb2z\" (UID: \"609f4320-8b1b-459e-9e9b-9aeeed31dbe2\") " pod="kube-system/global-pull-secret-syncer-phb2z"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791226    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-systemd\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791247    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-host\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.791260    4926 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791294    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-cni-multus\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791295    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-etc-systemd\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:24.791316    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs podName:d78ab9e2-f553-441e-9f3d-6d8985085c89 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:25.29130332 +0000 UTC m=+1.876843561 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs") pod "network-metrics-daemon-65ph6" (UID: "d78ab9e2-f553-441e-9f3d-6d8985085c89") : secret "metrics-daemon-secret" not found
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791334    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/65e4e478-1f58-498d-b6eb-619307c2943f-host\") pod \"tuned-z9phd\" (UID: \"65e4e478-1f58-498d-b6eb-619307c2943f\") " pod="openshift-cluster-node-tuning-operator/tuned-z9phd"
Oct 31 03:28:24.791502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.791334    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7ccdcc49-f072-4e15-b021-9bffc3fb674a-host-var-lib-cni-multus\") pod \"multus-c2qwl\" (UID: \"7ccdcc49-f072-4e15-b021-9bffc3fb674a\") " pod="openshift-multus/multus-c2qwl"
Oct 31 03:28:24.879258 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.879227    4926 scope.go:122] "RemoveContainer" containerID="1b470c024d032c0301032eaa971e8b3d98823c8fa645a9638b1a6a876ccea7ea"
Oct 31 03:28:24.895433 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:24.895384    4926 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 03:28:25.159368 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.159342    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:25.295959 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.295877    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:25.295959 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.295912    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:25.295959 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.295946    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:25.296241 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296028    4926 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 03:28:25.296241 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.296053    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:25.296241 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296085    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert podName:f4b87fa6-eceb-4783-b2c5-4685e08ba9ef nodeName:}" failed. No retries permitted until 2026-10-31 03:28:26.296071014 +0000 UTC m=+2.881611269 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k2fcf" (UID: "f4b87fa6-eceb-4783-b2c5-4685e08ba9ef") : secret "networking-console-plugin-cert" not found
Oct 31 03:28:25.296241 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296032    4926 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:25.296241 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296123    4926 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 03:28:25.296241 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296130    4926 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb458f74f-7bjpx: secret "image-registry-tls" not found
Oct 31 03:28:25.296241 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.296120    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:25.296241 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296166    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls podName:cdbd24b7-07ab-44ba-a564-0b84466b7bfa nodeName:}" failed. No retries permitted until 2026-10-31 03:28:26.296155294 +0000 UTC m=+2.881695563 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-cwnbr" (UID: "cdbd24b7-07ab-44ba-a564-0b84466b7bfa") : secret "samples-operator-tls" not found
Oct 31 03:28:25.296241 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296188    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls podName:260390fb-c26d-4989-9e8a-a104c398e1c6 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:26.296181526 +0000 UTC m=+2.881721766 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls") pod "image-registry-5cb458f74f-7bjpx" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6") : secret "image-registry-tls" not found
Oct 31 03:28:25.296241 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.296219    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.296295    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296303    4926 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296315    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:26.296306047 +0000 UTC m=+2.881846287 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : configmap references non-existent config key: service-ca.crt
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296337    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls podName:5615d56d-7e1b-4f88-803c-b6a1c0435263 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:26.296327674 +0000 UTC m=+2.881867914 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tw5k4" (UID: "5615d56d-7e1b-4f88-803c-b6a1c0435263") : secret "insights-runtime-extractor-tls" not found
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296368    4926 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296403    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs podName:d78ab9e2-f553-441e-9f3d-6d8985085c89 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:26.296390764 +0000 UTC m=+2.881931040 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs") pod "network-metrics-daemon-65ph6" (UID: "d78ab9e2-f553-441e-9f3d-6d8985085c89") : secret "metrics-daemon-secret" not found
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.296369    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296419    4926 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.296440    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296446    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls podName:a52638f4-7a5b-45de-9054-d173d66c7521 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:26.296435179 +0000 UTC m=+2.881975419 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls") pod "dns-default-zcpmp" (UID: "a52638f4-7a5b-45de-9054-d173d66c7521") : secret "dns-default-metrics-tls" not found
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.296467    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296482    4926 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.296498    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296503    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:26.296497239 +0000 UTC m=+2.882037479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : secret "router-metrics-certs-default" not found
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.296535    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5615d56d-7e1b-4f88-803c-b6a1c0435263-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296545    4926 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296571    4926 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296580    4926 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bd6c7ddd-l84xn: secret "image-registry-tls" not found
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296614    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls podName:05c25677-4b67-4a79-bbca-25bcc3ef43f1 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:26.29660199 +0000 UTC m=+2.882142251 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls") pod "image-registry-8bd6c7ddd-l84xn" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1") : secret "image-registry-tls" not found
Oct 31 03:28:25.296721 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.296627    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls podName:aef16625-d836-42cb-bf49-176f80e73474 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:26.296620244 +0000 UTC m=+2.882160483 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dwx2" (UID: "aef16625-d836-42cb-bf49-176f80e73474") : secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:25.587465 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.587394    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 03:28:25.587855 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.587788    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/1.log"
Oct 31 03:28:25.587855 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.587820    4926 generic.go:356] "Generic (PLEG): container finished" podID="bb5340b5-c677-40d1-a448-498561c62a17" containerID="f878ced68241c49d592f1d7545b4261a1f0ea616901999f57b62dc3e36a196eb" exitCode=255
Oct 31 03:28:25.587949 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.587930    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm" event={"ID":"bb5340b5-c677-40d1-a448-498561c62a17","Type":"ContainerDied","Data":"f878ced68241c49d592f1d7545b4261a1f0ea616901999f57b62dc3e36a196eb"}
Oct 31 03:28:25.588019 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.588007    4926 scope.go:122] "RemoveContainer" containerID="1b470c024d032c0301032eaa971e8b3d98823c8fa645a9638b1a6a876ccea7ea"
Oct 31 03:28:25.588618 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:25.588598    4926 scope.go:122] "RemoveContainer" containerID="f878ced68241c49d592f1d7545b4261a1f0ea616901999f57b62dc3e36a196eb"
Oct 31 03:28:25.588822 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:25.588788    4926 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-wpxfm_openshift-console-operator(bb5340b5-c677-40d1-a448-498561c62a17)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm" podUID="bb5340b5-c677-40d1-a448-498561c62a17"
Oct 31 03:28:26.306419 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:26.306375    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:26.306590 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:26.306433    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:26.306590 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:26.306477    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:26.306590 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:26.306505    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:26.306590 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:26.306539    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:26.306590 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306549    4926 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 03:28:26.306590 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:26.306562    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:26.306603    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306613    4926 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306629    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls podName:cdbd24b7-07ab-44ba-a564-0b84466b7bfa nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.306607826 +0000 UTC m=+4.892148089 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-cwnbr" (UID: "cdbd24b7-07ab-44ba-a564-0b84466b7bfa") : secret "samples-operator-tls" not found
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:26.306669    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306686    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls podName:5615d56d-7e1b-4f88-803c-b6a1c0435263 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.306673871 +0000 UTC m=+4.892214116 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tw5k4" (UID: "5615d56d-7e1b-4f88-803c-b6a1c0435263") : secret "insights-runtime-extractor-tls" not found
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306695    4926 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306725    4926 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:26.306732    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306552    4926 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306758    4926 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb458f74f-7bjpx: secret "image-registry-tls" not found
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306748    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.306734715 +0000 UTC m=+4.892274955 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : secret "router-metrics-certs-default" not found
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306790    4926 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306799    4926 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bd6c7ddd-l84xn: secret "image-registry-tls" not found
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306805    4926 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306810    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls podName:aef16625-d836-42cb-bf49-176f80e73474 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.306798756 +0000 UTC m=+4.892339000 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dwx2" (UID: "aef16625-d836-42cb-bf49-176f80e73474") : secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:26.306828 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306831    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls podName:260390fb-c26d-4989-9e8a-a104c398e1c6 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.306821981 +0000 UTC m=+4.892362221 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls") pod "image-registry-5cb458f74f-7bjpx" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6") : secret "image-registry-tls" not found
Oct 31 03:28:26.307451 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306847    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls podName:05c25677-4b67-4a79-bbca-25bcc3ef43f1 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.306839579 +0000 UTC m=+4.892379825 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls") pod "image-registry-8bd6c7ddd-l84xn" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1") : secret "image-registry-tls" not found
Oct 31 03:28:26.307451 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306863    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs podName:d78ab9e2-f553-441e-9f3d-6d8985085c89 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.306855446 +0000 UTC m=+4.892395686 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs") pod "network-metrics-daemon-65ph6" (UID: "d78ab9e2-f553-441e-9f3d-6d8985085c89") : secret "metrics-daemon-secret" not found
Oct 31 03:28:26.307451 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306879    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.3068701 +0000 UTC m=+4.892410342 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : configmap references non-existent config key: service-ca.crt
Oct 31 03:28:26.307451 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306883    4926 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 03:28:26.307451 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.306913    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls podName:a52638f4-7a5b-45de-9054-d173d66c7521 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.306900467 +0000 UTC m=+4.892440711 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls") pod "dns-default-zcpmp" (UID: "a52638f4-7a5b-45de-9054-d173d66c7521") : secret "dns-default-metrics-tls" not found
Oct 31 03:28:26.307451 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:26.306936    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:26.307451 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.307037    4926 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 03:28:26.307451 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:26.307072    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert podName:f4b87fa6-eceb-4783-b2c5-4685e08ba9ef nodeName:}" failed. No retries permitted until 2026-10-31 03:28:28.307061044 +0000 UTC m=+4.892601284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k2fcf" (UID: "f4b87fa6-eceb-4783-b2c5-4685e08ba9ef") : secret "networking-console-plugin-cert" not found
Oct 31 03:28:26.593440 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:26.593365    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 03:28:26.594834 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:26.594803    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-mft5z" event={"ID":"945a916b-41c5-46f6-9c1a-9a8c93f7d6d2","Type":"ContainerStarted","Data":"50ef6c4fe4036d67486993af3c4a4094caa14a8f944dce044b57c73f204e1e11"}
Oct 31 03:28:28.324860 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:28.324822    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:28.325288 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:28.324868    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:28.325288 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.324968    4926 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 03:28:28.325288 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.324982    4926 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 03:28:28.325288 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:28.324994    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:28.325288 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325029    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:32.325011393 +0000 UTC m=+8.910551635 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : secret "router-metrics-certs-default" not found
Oct 31 03:28:28.325288 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325051    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls podName:a52638f4-7a5b-45de-9054-d173d66c7521 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:32.32504115 +0000 UTC m=+8.910581392 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls") pod "dns-default-zcpmp" (UID: "a52638f4-7a5b-45de-9054-d173d66c7521") : secret "dns-default-metrics-tls" not found
Oct 31 03:28:28.325288 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325067    4926 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:28.325288 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:28.325099    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:28.325288 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325116    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls podName:aef16625-d836-42cb-bf49-176f80e73474 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:32.325104514 +0000 UTC m=+8.910644772 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-2dwx2" (UID: "aef16625-d836-42cb-bf49-176f80e73474") : secret "cluster-monitoring-operator-tls" not found
Oct 31 03:28:28.325288 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325160    4926 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:28.325288 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325170    4926 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bd6c7ddd-l84xn: secret "image-registry-tls" not found
Oct 31 03:28:28.325288 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325200    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls podName:05c25677-4b67-4a79-bbca-25bcc3ef43f1 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:32.325190872 +0000 UTC m=+8.910731126 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls") pod "image-registry-8bd6c7ddd-l84xn" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1") : secret "image-registry-tls" not found
Oct 31 03:28:28.325288 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325207    4926 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 31 03:28:28.325288 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325234    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert podName:f4b87fa6-eceb-4783-b2c5-4685e08ba9ef nodeName:}" failed. No retries permitted until 2026-10-31 03:28:32.325227655 +0000 UTC m=+8.910767894 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-k2fcf" (UID: "f4b87fa6-eceb-4783-b2c5-4685e08ba9ef") : secret "networking-console-plugin-cert" not found
Oct 31 03:28:28.325288 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:28.325156    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:28.325847 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:28.325350    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:28.325847 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:28.325383    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:28.325847 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:28.325422    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:28.325847 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:28.325450    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:28.325847 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325467    4926 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:28.325847 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325478    4926 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb458f74f-7bjpx: secret "image-registry-tls" not found
Oct 31 03:28:28.325847 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:28.325486    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:28.325847 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325523    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls podName:260390fb-c26d-4989-9e8a-a104c398e1c6 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:32.325506649 +0000 UTC m=+8.911046889 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls") pod "image-registry-5cb458f74f-7bjpx" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6") : secret "image-registry-tls" not found
Oct 31 03:28:28.325847 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325554    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:32.325544078 +0000 UTC m=+8.911084351 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : configmap references non-existent config key: service-ca.crt
Oct 31 03:28:28.325847 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325561    4926 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 31 03:28:28.325847 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325578    4926 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 31 03:28:28.325847 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325587    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls podName:5615d56d-7e1b-4f88-803c-b6a1c0435263 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:32.325581426 +0000 UTC m=+8.911121667 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tw5k4" (UID: "5615d56d-7e1b-4f88-803c-b6a1c0435263") : secret "insights-runtime-extractor-tls" not found
Oct 31 03:28:28.325847 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325593    4926 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 03:28:28.325847 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325614    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs podName:d78ab9e2-f553-441e-9f3d-6d8985085c89 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:32.325605576 +0000 UTC m=+8.911145841 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs") pod "network-metrics-daemon-65ph6" (UID: "d78ab9e2-f553-441e-9f3d-6d8985085c89") : secret "metrics-daemon-secret" not found
Oct 31 03:28:28.325847 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:28.325671    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls podName:cdbd24b7-07ab-44ba-a564-0b84466b7bfa nodeName:}" failed. No retries permitted until 2026-10-31 03:28:32.325654699 +0000 UTC m=+8.911194953 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-cwnbr" (UID: "cdbd24b7-07ab-44ba-a564-0b84466b7bfa") : secret "samples-operator-tls" not found
Oct 31 03:28:29.868422 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:29.868375    4926 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:29.868805 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:29.868734    4926 scope.go:122] "RemoveContainer" containerID="f878ced68241c49d592f1d7545b4261a1f0ea616901999f57b62dc3e36a196eb"
Oct 31 03:28:29.868920 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:29.868902    4926 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-wpxfm_openshift-console-operator(bb5340b5-c677-40d1-a448-498561c62a17)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm" podUID="bb5340b5-c677-40d1-a448-498561c62a17"
Oct 31 03:28:30.578869 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:30.578835    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:30.606214 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:30.606191    4926 scope.go:122] "RemoveContainer" containerID="f878ced68241c49d592f1d7545b4261a1f0ea616901999f57b62dc3e36a196eb"
Oct 31 03:28:30.606407 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:30.606388    4926 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-wpxfm_openshift-console-operator(bb5340b5-c677-40d1-a448-498561c62a17)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm" podUID="bb5340b5-c677-40d1-a448-498561c62a17"
Oct 31 03:28:30.634720 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:30.634695    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:28:31.586261 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:31.586235    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:31.586679 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:31.586521    4926 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 31 03:28:32.365988 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.365945    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:32.366166 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.366006    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:32.366166 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.366039    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:32.366166 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.366075    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:32.366166 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:32.366109    4926 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 31 03:28:32.366166 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:32.366131    4926 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 31 03:28:32.366448 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:32.366171    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls podName:a52638f4-7a5b-45de-9054-d173d66c7521 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:40.366156892 +0000 UTC m=+16.951697135 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls") pod "dns-default-zcpmp" (UID: "a52638f4-7a5b-45de-9054-d173d66c7521") : secret "dns-default-metrics-tls" not found
Oct 31 03:28:32.366448 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:32.366210    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:40.366193248 +0000 UTC m=+16.951733507 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : secret "router-metrics-certs-default" not found
Oct 31 03:28:32.366448 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.366110    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:32.366448 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:32.366229    4926 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:32.366448 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:32.366245    4926 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bd6c7ddd-l84xn: secret "image-registry-tls" not found
Oct 31 03:28:32.366448 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.366315    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:32.366448 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:32.366325    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls podName:05c25677-4b67-4a79-bbca-25bcc3ef43f1 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:40.366312046 +0000 UTC m=+16.951852293 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls") pod "image-registry-8bd6c7ddd-l84xn" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1") : secret "image-registry-tls" not found
Oct 31 03:28:32.366448 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.366364    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:32.366448 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:32.366387    4926 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 31 03:28:32.366448 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:32.366398    4926 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cb458f74f-7bjpx: secret "image-registry-tls" not found
Oct 31 03:28:32.366448 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.366405    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:32.366448 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:32.366430    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls podName:260390fb-c26d-4989-9e8a-a104c398e1c6 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:40.366419866 +0000 UTC m=+16.951960109 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls") pod "image-registry-5cb458f74f-7bjpx" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6") : secret "image-registry-tls" not found
Oct 31 03:28:32.366448 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.366448    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:32.367003 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:32.366465    4926 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 31 03:28:32.367003 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.366492    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:32.367003 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.366511    4926 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 31 03:28:32.367003 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:32.366527    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls podName:cdbd24b7-07ab-44ba-a564-0b84466b7bfa nodeName:}" failed. No retries permitted until 2026-10-31 03:28:40.366512833 +0000 UTC m=+16.952053088 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-cwnbr" (UID: "cdbd24b7-07ab-44ba-a564-0b84466b7bfa") : secret "samples-operator-tls" not found
Oct 31 03:28:32.367003 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:28:32.366560    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle podName:26a79eda-be35-4aa8-b10b-cb2b1133a086 nodeName:}" failed. No retries permitted until 2026-10-31 03:28:40.366549751 +0000 UTC m=+16.952089992 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle") pod "router-default-77685b76bf-cmqhp" (UID: "26a79eda-be35-4aa8-b10b-cb2b1133a086") : configmap references non-existent config key: service-ca.crt
Oct 31 03:28:32.368813 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.368792    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d78ab9e2-f553-441e-9f3d-6d8985085c89-metrics-certs\") pod \"network-metrics-daemon-65ph6\" (UID: \"d78ab9e2-f553-441e-9f3d-6d8985085c89\") " pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:32.369227 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.369202    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5615d56d-7e1b-4f88-803c-b6a1c0435263-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tw5k4\" (UID: \"5615d56d-7e1b-4f88-803c-b6a1c0435263\") " pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:32.369306 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.368915    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f4b87fa6-eceb-4783-b2c5-4685e08ba9ef-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-k2fcf\" (UID: \"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:32.369357 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.369322    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aef16625-d836-42cb-bf49-176f80e73474-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-2dwx2\" (UID: \"aef16625-d836-42cb-bf49-176f80e73474\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:32.380241 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.380220    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"
Oct 31 03:28:32.381724 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.381712    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"
Oct 31 03:28:32.387340 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.387322    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-65ph6"
Oct 31 03:28:32.394597 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.394575    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tw5k4"
Oct 31 03:28:32.756255 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.756223    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2"]
Oct 31 03:28:32.792989 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:32.792962    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaef16625_d836_42cb_bf49_176f80e73474.slice/crio-5267b548755cfdd7d85e030e5db384cd7a7c7111e91c620817cbc15c077ad3cd WatchSource:0}: Error finding container 5267b548755cfdd7d85e030e5db384cd7a7c7111e91c620817cbc15c077ad3cd: Status 404 returned error can't find the container with id 5267b548755cfdd7d85e030e5db384cd7a7c7111e91c620817cbc15c077ad3cd
Oct 31 03:28:32.801846 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.801827    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-k2fcf"]
Oct 31 03:28:32.804136 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:32.804113    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4b87fa6_eceb_4783_b2c5_4685e08ba9ef.slice/crio-393b1badce2d931da09b10d064ab7f8e4f0906f99063c22de6b088f36993ccce WatchSource:0}: Error finding container 393b1badce2d931da09b10d064ab7f8e4f0906f99063c22de6b088f36993ccce: Status 404 returned error can't find the container with id 393b1badce2d931da09b10d064ab7f8e4f0906f99063c22de6b088f36993ccce
Oct 31 03:28:32.805463 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.805447    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-65ph6"]
Oct 31 03:28:32.805911 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.805873    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tw5k4"]
Oct 31 03:28:32.839140 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:32.839113    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5615d56d_7e1b_4f88_803c_b6a1c0435263.slice/crio-634f2f84816837b860631295a1be2ae27fd1f73697c2f3ba4cb050d41e0450f3 WatchSource:0}: Error finding container 634f2f84816837b860631295a1be2ae27fd1f73697c2f3ba4cb050d41e0450f3: Status 404 returned error can't find the container with id 634f2f84816837b860631295a1be2ae27fd1f73697c2f3ba4cb050d41e0450f3
Oct 31 03:28:32.839856 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:32.839836    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78ab9e2_f553_441e_9f3d_6d8985085c89.slice/crio-352de033794dea5c95b0e4d0c2b61f2e3187ecd7da5351e0a0dc5349d545a8c1 WatchSource:0}: Error finding container 352de033794dea5c95b0e4d0c2b61f2e3187ecd7da5351e0a0dc5349d545a8c1: Status 404 returned error can't find the container with id 352de033794dea5c95b0e4d0c2b61f2e3187ecd7da5351e0a0dc5349d545a8c1
Oct 31 03:28:32.933763 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:32.933740    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-g95zb_694e1b6d-1615-4134-b262-e9df58af99b5/dns-node-resolver/0.log"
Oct 31 03:28:33.615664 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:33.615626    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2" event={"ID":"aef16625-d836-42cb-bf49-176f80e73474","Type":"ContainerStarted","Data":"5267b548755cfdd7d85e030e5db384cd7a7c7111e91c620817cbc15c077ad3cd"}
Oct 31 03:28:33.616718 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:33.616689    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-65ph6" event={"ID":"d78ab9e2-f553-441e-9f3d-6d8985085c89","Type":"ContainerStarted","Data":"352de033794dea5c95b0e4d0c2b61f2e3187ecd7da5351e0a0dc5349d545a8c1"}
Oct 31 03:28:33.618099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:33.618074    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tw5k4" event={"ID":"5615d56d-7e1b-4f88-803c-b6a1c0435263","Type":"ContainerStarted","Data":"699faa0e08f944cdb8dec90525ba8c17c9439cf4bcfdaa883e2b341032a34ca2"}
Oct 31 03:28:33.618190 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:33.618119    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tw5k4" event={"ID":"5615d56d-7e1b-4f88-803c-b6a1c0435263","Type":"ContainerStarted","Data":"634f2f84816837b860631295a1be2ae27fd1f73697c2f3ba4cb050d41e0450f3"}
Oct 31 03:28:33.619137 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:33.619119    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf" event={"ID":"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef","Type":"ContainerStarted","Data":"393b1badce2d931da09b10d064ab7f8e4f0906f99063c22de6b088f36993ccce"}
Oct 31 03:28:34.133152 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:34.133124    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-fd7k8_5f03b006-c2d0-4e6b-920b-88d146280084/node-ca/0.log"
Oct 31 03:28:34.533497 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:34.533464    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pq8vh_fc6215a3-f0b6-4f89-9a0d-30589bb43c5a/serve-healthcheck-canary/0.log"
Oct 31 03:28:34.624625 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:34.624593    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tw5k4" event={"ID":"5615d56d-7e1b-4f88-803c-b6a1c0435263","Type":"ContainerStarted","Data":"2302ebe87dddff80aed7ad5b84aad3d108a73e33a1477b6a28ad4be7a92c5802"}
Oct 31 03:28:35.134882 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:35.134796    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-mft5z_945a916b-41c5-46f6-9c1a-9a8c93f7d6d2/migrator/0.log"
Oct 31 03:28:35.335541 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:35.335510    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg_7cc09f0c-627f-4445-b172-9556cab2bbdd/kube-storage-version-migrator-operator/0.log"
Oct 31 03:28:35.534681 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:35.534654    4926 scope.go:122] "RemoveContainer" containerID="f878ced68241c49d592f1d7545b4261a1f0ea616901999f57b62dc3e36a196eb"
Oct 31 03:28:35.630458 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:35.630421    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-65ph6" event={"ID":"d78ab9e2-f553-441e-9f3d-6d8985085c89","Type":"ContainerStarted","Data":"f41faa7af04efe519b778ca55f9f4359dff91598089d4f909782553dc13ee091"}
Oct 31 03:28:35.630626 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:35.630489    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-65ph6" event={"ID":"d78ab9e2-f553-441e-9f3d-6d8985085c89","Type":"ContainerStarted","Data":"7bd0072544d49929dd67fa96bb560e0baaf67bb987f7c7abc34a9287a2ec978d"}
Oct 31 03:28:35.633823 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:35.633792    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-k2fcf" event={"ID":"f4b87fa6-eceb-4783-b2c5-4685e08ba9ef","Type":"ContainerStarted","Data":"7b4c37fed24a70bd9800b753182d5d8ad38ef29eff32e52c8f068193c12335b5"}
Oct 31 03:28:35.636553 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:35.636513    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 03:28:35.636553 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:35.636556    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm" event={"ID":"bb5340b5-c677-40d1-a448-498561c62a17","Type":"ContainerStarted","Data":"6c8622655aca24a7c0166768b8cacf94ec657b2261e7ff40755a8aa9b8f320c9"}
Oct 31 03:28:35.637193 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:35.637176    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:35.638563 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:35.638532    4926 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-wpxfm container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.20:8443/readyz\": dial tcp 10.132.0.20:8443: connect: connection refused" start-of-body=
Oct 31 03:28:35.638648 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:35.638580    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm" podUID="bb5340b5-c677-40d1-a448-498561c62a17" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.20:8443/readyz\": dial tcp 10.132.0.20:8443: connect: connection refused"
Oct 31 03:28:36.596679 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:36.596623    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wwpg9"
Oct 31 03:28:36.642016 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:36.641986    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-2dwx2" event={"ID":"aef16625-d836-42cb-bf49-176f80e73474","Type":"ContainerStarted","Data":"278f8c1afe00558f00eab1fc0a41e1ad1d1621aa4910b63a16218a9bb143ed73"}
Oct 31 03:28:36.647025 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:36.647003    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-wpxfm"
Oct 31 03:28:37.649177 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:37.649136    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tw5k4" event={"ID":"5615d56d-7e1b-4f88-803c-b6a1c0435263","Type":"ContainerStarted","Data":"b937f0538b00d3ad427506ac0010b781791f89c54f104a63c5edbd0b9d3396b8"}
Oct 31 03:28:40.445153 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.445115    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:40.445698 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.445172    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:40.445698 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.445221    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:40.445698 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.445301    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:40.445698 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.445340    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:40.445698 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.445493    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:40.446179 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.446154    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/26a79eda-be35-4aa8-b10b-cb2b1133a086-service-ca-bundle\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:40.448395 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.448370    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/26a79eda-be35-4aa8-b10b-cb2b1133a086-metrics-certs\") pod \"router-default-77685b76bf-cmqhp\" (UID: \"26a79eda-be35-4aa8-b10b-cb2b1133a086\") " pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:40.448480 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.448434    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") pod \"image-registry-5cb458f74f-7bjpx\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") " pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:40.448480 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.448466    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/cdbd24b7-07ab-44ba-a564-0b84466b7bfa-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-cwnbr\" (UID: \"cdbd24b7-07ab-44ba-a564-0b84466b7bfa\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:40.448480 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.448474    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") pod \"image-registry-8bd6c7ddd-l84xn\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") " pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:40.448664 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.448650    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a52638f4-7a5b-45de-9054-d173d66c7521-metrics-tls\") pod \"dns-default-zcpmp\" (UID: \"a52638f4-7a5b-45de-9054-d173d66c7521\") " pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:40.478548 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.478518    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:40.479560 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.479544    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"
Oct 31 03:28:40.481753 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.481731    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:40.491386 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.491364    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:40.493519 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.493498    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:40.911987 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.911961    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-77685b76bf-cmqhp"]
Oct 31 03:28:40.914448 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:40.914418    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26a79eda_be35_4aa8_b10b_cb2b1133a086.slice/crio-cae96c967be8d133f5b9584be2d2c4afa0231b8b77e3b5d4ce2921a736e6d4e0 WatchSource:0}: Error finding container cae96c967be8d133f5b9584be2d2c4afa0231b8b77e3b5d4ce2921a736e6d4e0: Status 404 returned error can't find the container with id cae96c967be8d133f5b9584be2d2c4afa0231b8b77e3b5d4ce2921a736e6d4e0
Oct 31 03:28:40.960401 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.960374    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr"]
Oct 31 03:28:40.960540 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:40.960524    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8bd6c7ddd-l84xn"]
Oct 31 03:28:40.969404 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:40.969380    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05c25677_4b67_4a79_bbca_25bcc3ef43f1.slice/crio-38b42fd4d1ff5cea3fc9ebb176aa92ed8a0dd4bc8065a42fe7abd36fbdef2682 WatchSource:0}: Error finding container 38b42fd4d1ff5cea3fc9ebb176aa92ed8a0dd4bc8065a42fe7abd36fbdef2682: Status 404 returned error can't find the container with id 38b42fd4d1ff5cea3fc9ebb176aa92ed8a0dd4bc8065a42fe7abd36fbdef2682
Oct 31 03:28:41.009813 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:41.009787    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cb458f74f-7bjpx"]
Oct 31 03:28:41.010379 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:41.010358    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zcpmp"]
Oct 31 03:28:41.664028 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:41.663993    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn" event={"ID":"05c25677-4b67-4a79-bbca-25bcc3ef43f1","Type":"ContainerStarted","Data":"56aa3f2123e2e8ac2976d2279563ddbfc14a45445853e0ed9bb9c3e81f4800da"}
Oct 31 03:28:41.664476 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:41.664046    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn" event={"ID":"05c25677-4b67-4a79-bbca-25bcc3ef43f1","Type":"ContainerStarted","Data":"38b42fd4d1ff5cea3fc9ebb176aa92ed8a0dd4bc8065a42fe7abd36fbdef2682"}
Oct 31 03:28:41.664476 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:41.664101    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:28:41.665659 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:41.665629    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx" event={"ID":"260390fb-c26d-4989-9e8a-a104c398e1c6","Type":"ContainerStarted","Data":"ad805f92fbd3eaf50f40accf7fc8f07da7deb9e86875ff5d197dd8ec96a5072b"}
Oct 31 03:28:41.665814 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:41.665673    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx" event={"ID":"260390fb-c26d-4989-9e8a-a104c398e1c6","Type":"ContainerStarted","Data":"f841ebac49106ddfeaf8c766d7991fd08ac72751650df0f40145cf422767aa00"}
Oct 31 03:28:41.665814 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:41.665760    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:28:41.666799 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:41.666776    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr" event={"ID":"cdbd24b7-07ab-44ba-a564-0b84466b7bfa","Type":"ContainerStarted","Data":"2a5e0620f15a3235282f2c1630028b5cc694f8920864c6b066bd4050c71b2310"}
Oct 31 03:28:41.667875 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:41.667857    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zcpmp" event={"ID":"a52638f4-7a5b-45de-9054-d173d66c7521","Type":"ContainerStarted","Data":"16c4e68cfe4ccdaa81fb5ee302b10e9bee8ea01766950b87c0d6549662453f87"}
Oct 31 03:28:41.669307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:41.669282    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77685b76bf-cmqhp" event={"ID":"26a79eda-be35-4aa8-b10b-cb2b1133a086","Type":"ContainerStarted","Data":"5952df44229dd4409f2e832075d98c5b94bd6f93df725496b373bc5d6fd43a34"}
Oct 31 03:28:41.669384 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:41.669319    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-77685b76bf-cmqhp" event={"ID":"26a79eda-be35-4aa8-b10b-cb2b1133a086","Type":"ContainerStarted","Data":"cae96c967be8d133f5b9584be2d2c4afa0231b8b77e3b5d4ce2921a736e6d4e0"}
Oct 31 03:28:42.478806 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:42.478773    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:42.481429 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:42.481409    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:42.673599 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:42.673562    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:42.674885 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:42.674863    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-77685b76bf-cmqhp"
Oct 31 03:28:43.684573 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:43.684469    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zcpmp" event={"ID":"a52638f4-7a5b-45de-9054-d173d66c7521","Type":"ContainerStarted","Data":"21b7c48c50eb0078889f87388e2c5814f511dd5e133594639ffd72a6d47b4b88"}
Oct 31 03:28:43.684958 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:43.684570    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zcpmp" event={"ID":"a52638f4-7a5b-45de-9054-d173d66c7521","Type":"ContainerStarted","Data":"afc81e2dd647bf712aaa39558567bccda7a74fee1e3cc3ec6a65b589c031e6e2"}
Oct 31 03:28:43.754655 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:43.754619    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cb458f74f-7bjpx"]
Oct 31 03:28:44.690046 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:44.690001    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr" event={"ID":"cdbd24b7-07ab-44ba-a564-0b84466b7bfa","Type":"ContainerStarted","Data":"7fd77623d2405fe95d81484d86435b22813f5b5efbd82fe13d2da2104e7fa52a"}
Oct 31 03:28:44.690497 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:44.690068    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-cwnbr" event={"ID":"cdbd24b7-07ab-44ba-a564-0b84466b7bfa","Type":"ContainerStarted","Data":"11875e1d5c82adf6b2e3f62230402c647f2891572be7e35c53e916ca165b0448"}
Oct 31 03:28:44.690497 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:44.690437    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:48.740031 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:48.740000    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qwh92"
Oct 31 03:28:54.696727 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:54.696687    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zcpmp"
Oct 31 03:28:55.575564 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.575529    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-glhsr"]
Oct 31 03:28:55.628390 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.628361    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.726851 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.726824    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 31 03:28:55.726851 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.726841    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 31 03:28:55.727324 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.726881    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 31 03:28:55.727324 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.726910    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-b9rvf\""
Oct 31 03:28:55.727324 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.727015    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 31 03:28:55.768725 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.768701    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b801defd-0f62-417c-b440-f5c8e8635701-node-exporter-tls\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.768858 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.768727    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b801defd-0f62-417c-b440-f5c8e8635701-node-exporter-textfile\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.768858 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.768747    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b801defd-0f62-417c-b440-f5c8e8635701-node-exporter-wtmp\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.768858 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.768764    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b801defd-0f62-417c-b440-f5c8e8635701-node-exporter-accelerators-collector-config\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.768858 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.768806    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b801defd-0f62-417c-b440-f5c8e8635701-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.769008 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.768883    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b801defd-0f62-417c-b440-f5c8e8635701-sys\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.769008 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.768902    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b801defd-0f62-417c-b440-f5c8e8635701-metrics-client-ca\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.769008 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.768930    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b801defd-0f62-417c-b440-f5c8e8635701-root\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.769008 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.768950    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkrvf\" (UniqueName: \"kubernetes.io/projected/b801defd-0f62-417c-b440-f5c8e8635701-kube-api-access-hkrvf\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.870285 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.870199    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkrvf\" (UniqueName: \"kubernetes.io/projected/b801defd-0f62-417c-b440-f5c8e8635701-kube-api-access-hkrvf\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.870285 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.870244    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b801defd-0f62-417c-b440-f5c8e8635701-node-exporter-tls\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.870476 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.870289    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b801defd-0f62-417c-b440-f5c8e8635701-node-exporter-textfile\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.870476 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.870315    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b801defd-0f62-417c-b440-f5c8e8635701-node-exporter-wtmp\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.870476 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.870339    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b801defd-0f62-417c-b440-f5c8e8635701-node-exporter-accelerators-collector-config\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.870476 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.870368    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b801defd-0f62-417c-b440-f5c8e8635701-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.870476 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.870413    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b801defd-0f62-417c-b440-f5c8e8635701-sys\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.870476 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.870434    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b801defd-0f62-417c-b440-f5c8e8635701-metrics-client-ca\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.870476 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.870475    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b801defd-0f62-417c-b440-f5c8e8635701-root\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.870826 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.870508    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b801defd-0f62-417c-b440-f5c8e8635701-node-exporter-wtmp\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.870826 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.870540    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b801defd-0f62-417c-b440-f5c8e8635701-root\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.870826 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.870546    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b801defd-0f62-417c-b440-f5c8e8635701-sys\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.870826 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.870750    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b801defd-0f62-417c-b440-f5c8e8635701-node-exporter-textfile\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.871064 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.871042    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b801defd-0f62-417c-b440-f5c8e8635701-node-exporter-accelerators-collector-config\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.871327 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.871307    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b801defd-0f62-417c-b440-f5c8e8635701-metrics-client-ca\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.872755 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.872733    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b801defd-0f62-417c-b440-f5c8e8635701-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.872845 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.872823    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b801defd-0f62-417c-b440-f5c8e8635701-node-exporter-tls\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.929431 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.929410    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkrvf\" (UniqueName: \"kubernetes.io/projected/b801defd-0f62-417c-b440-f5c8e8635701-kube-api-access-hkrvf\") pod \"node-exporter-glhsr\" (UID: \"b801defd-0f62-417c-b440-f5c8e8635701\") " pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:55.979695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:55.979666    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-glhsr"
Oct 31 03:28:56.014914 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:28:56.014882    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb801defd_0f62_417c_b440_f5c8e8635701.slice/crio-b650a0139e689ec1905ae6cf664663da4d7d2d3d2354dcbbd2b6667e196e1da8 WatchSource:0}: Error finding container b650a0139e689ec1905ae6cf664663da4d7d2d3d2354dcbbd2b6667e196e1da8: Status 404 returned error can't find the container with id b650a0139e689ec1905ae6cf664663da4d7d2d3d2354dcbbd2b6667e196e1da8
Oct 31 03:28:56.735636 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:56.735599    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-glhsr" event={"ID":"b801defd-0f62-417c-b440-f5c8e8635701","Type":"ContainerStarted","Data":"b650a0139e689ec1905ae6cf664663da4d7d2d3d2354dcbbd2b6667e196e1da8"}
Oct 31 03:28:58.746241 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:58.746202    4926 generic.go:356] "Generic (PLEG): container finished" podID="b801defd-0f62-417c-b440-f5c8e8635701" containerID="ebe93ef49ea63451d976abbfc750c843112116a0f5a9de29e6542d9dafa54373" exitCode=0
Oct 31 03:28:58.746687 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:58.746300    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-glhsr" event={"ID":"b801defd-0f62-417c-b440-f5c8e8635701","Type":"ContainerDied","Data":"ebe93ef49ea63451d976abbfc750c843112116a0f5a9de29e6542d9dafa54373"}
Oct 31 03:28:59.752405 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:59.752369    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-glhsr" event={"ID":"b801defd-0f62-417c-b440-f5c8e8635701","Type":"ContainerStarted","Data":"1740781b56e8fd861080aabbda85e0fc8103ca799eeb467e6dfc2d5204a7429e"}
Oct 31 03:28:59.752800 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:59.752424    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-glhsr" event={"ID":"b801defd-0f62-417c-b440-f5c8e8635701","Type":"ContainerStarted","Data":"aee3951c4c50470d2e95d019002353d353f9de1bbd858485f81f8fdfba515e7c"}
Oct 31 03:28:59.914379 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:28:59.914316    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-glhsr" podStartSLOduration=2.636772799 podStartE2EDuration="4.914300962s" totalImagesPullingTime="2.19282991s" totalInitContainerRuntime="84.698253ms" isStatefulPod=true podCreationTimestamp="2026-10-31 03:28:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:28:59.913740709 +0000 UTC m=+36.499280970" watchObservedRunningTime="2026-10-31 03:28:59.914300962 +0000 UTC m=+36.499841264"
Oct 31 03:29:00.495426 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:00.495394    4926 patch_prober.go:29] interesting pod/image-registry-8bd6c7ddd-l84xn container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 31 03:29:00.495583 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:00.495443    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn" podUID="05c25677-4b67-4a79-bbca-25bcc3ef43f1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 03:29:00.637625 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:00.637594    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-n6jzp"
Oct 31 03:29:02.677319 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:02.677235    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:29:03.760566 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:03.760537    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:29:05.312535 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:29:05.312498    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:29:06.456065 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:06.456030    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8bd6c7ddd-l84xn"]
Oct 31 03:29:08.778191 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:08.778143    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx" podUID="260390fb-c26d-4989-9e8a-a104c398e1c6" containerName="registry" containerID="cri-o://ad805f92fbd3eaf50f40accf7fc8f07da7deb9e86875ff5d197dd8ec96a5072b" gracePeriod=30
Oct 31 03:29:09.015240 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.015216    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:29:09.185236 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.185167    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/260390fb-c26d-4989-9e8a-a104c398e1c6-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/260390fb-c26d-4989-9e8a-a104c398e1c6-ca-trust-extracted\") pod \"260390fb-c26d-4989-9e8a-a104c398e1c6\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") "
Oct 31 03:29:09.185236 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.185206    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") pod \"260390fb-c26d-4989-9e8a-a104c398e1c6\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") "
Oct 31 03:29:09.185236 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.185233    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-installation-pull-secrets\") pod \"260390fb-c26d-4989-9e8a-a104c398e1c6\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") "
Oct 31 03:29:09.185501 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.185252    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-image-registry-private-configuration\") pod \"260390fb-c26d-4989-9e8a-a104c398e1c6\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") "
Oct 31 03:29:09.185501 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.185298    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-trusted-ca\") pod \"260390fb-c26d-4989-9e8a-a104c398e1c6\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") "
Oct 31 03:29:09.185501 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.185326    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-bound-sa-token\") pod \"260390fb-c26d-4989-9e8a-a104c398e1c6\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") "
Oct 31 03:29:09.185501 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.185361    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-kube-api-access-pfc5g\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-kube-api-access-pfc5g\") pod \"260390fb-c26d-4989-9e8a-a104c398e1c6\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") "
Oct 31 03:29:09.185501 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.185398    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-certificates\") pod \"260390fb-c26d-4989-9e8a-a104c398e1c6\" (UID: \"260390fb-c26d-4989-9e8a-a104c398e1c6\") "
Oct 31 03:29:09.186019 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.185930    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-trusted-ca" pod "260390fb-c26d-4989-9e8a-a104c398e1c6" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:29:09.186019 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.185958    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-certificates" pod "260390fb-c26d-4989-9e8a-a104c398e1c6" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:29:09.193983 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.193949    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-kube-api-access-pfc5g" pod "260390fb-c26d-4989-9e8a-a104c398e1c6" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6"). InnerVolumeSpecName "kube-api-access-pfc5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:29:09.193983 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.193971    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/260390fb-c26d-4989-9e8a-a104c398e1c6-ca-trust-extracted" pod "260390fb-c26d-4989-9e8a-a104c398e1c6" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:29:09.193983 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.193955    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls" pod "260390fb-c26d-4989-9e8a-a104c398e1c6" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:29:09.194216 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.194095    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-image-registry-private-configuration" pod "260390fb-c26d-4989-9e8a-a104c398e1c6" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:29:09.194216 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.194110    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-bound-sa-token" pod "260390fb-c26d-4989-9e8a-a104c398e1c6" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:29:09.194216 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.194144    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-installation-pull-secrets" pod "260390fb-c26d-4989-9e8a-a104c398e1c6" (UID: "260390fb-c26d-4989-9e8a-a104c398e1c6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:29:09.286664 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.286619    4926 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/260390fb-c26d-4989-9e8a-a104c398e1c6-ca-trust-extracted\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:09.286664 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.286656    4926 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-tls\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:09.286664 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.286666    4926 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-installation-pull-secrets\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:09.286885 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.286677    4926 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/260390fb-c26d-4989-9e8a-a104c398e1c6-image-registry-private-configuration\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:09.286885 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.286686    4926 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-trusted-ca\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:09.286885 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.286694    4926 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-bound-sa-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:09.286885 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.286704    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfc5g\" (UniqueName: \"kubernetes.io/projected/260390fb-c26d-4989-9e8a-a104c398e1c6-kube-api-access-pfc5g\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:09.286885 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.286712    4926 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/260390fb-c26d-4989-9e8a-a104c398e1c6-registry-certificates\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:09.792060 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.792028    4926 generic.go:356] "Generic (PLEG): container finished" podID="260390fb-c26d-4989-9e8a-a104c398e1c6" containerID="ad805f92fbd3eaf50f40accf7fc8f07da7deb9e86875ff5d197dd8ec96a5072b" exitCode=0
Oct 31 03:29:09.792517 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.792072    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx"
Oct 31 03:29:09.792517 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.792090    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx" event={"ID":"260390fb-c26d-4989-9e8a-a104c398e1c6","Type":"ContainerDied","Data":"ad805f92fbd3eaf50f40accf7fc8f07da7deb9e86875ff5d197dd8ec96a5072b"}
Oct 31 03:29:09.792517 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.792128    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cb458f74f-7bjpx" event={"ID":"260390fb-c26d-4989-9e8a-a104c398e1c6","Type":"ContainerDied","Data":"f841ebac49106ddfeaf8c766d7991fd08ac72751650df0f40145cf422767aa00"}
Oct 31 03:29:09.792517 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.792147    4926 scope.go:122] "RemoveContainer" containerID="ad805f92fbd3eaf50f40accf7fc8f07da7deb9e86875ff5d197dd8ec96a5072b"
Oct 31 03:29:09.801290 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.801242    4926 scope.go:122] "RemoveContainer" containerID="ad805f92fbd3eaf50f40accf7fc8f07da7deb9e86875ff5d197dd8ec96a5072b"
Oct 31 03:29:09.801729 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:29:09.801702    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad805f92fbd3eaf50f40accf7fc8f07da7deb9e86875ff5d197dd8ec96a5072b\": container with ID starting with ad805f92fbd3eaf50f40accf7fc8f07da7deb9e86875ff5d197dd8ec96a5072b not found: ID does not exist" containerID="ad805f92fbd3eaf50f40accf7fc8f07da7deb9e86875ff5d197dd8ec96a5072b"
Oct 31 03:29:09.801792 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.801742    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad805f92fbd3eaf50f40accf7fc8f07da7deb9e86875ff5d197dd8ec96a5072b"} err="failed to get container status \"ad805f92fbd3eaf50f40accf7fc8f07da7deb9e86875ff5d197dd8ec96a5072b\": rpc error: code = NotFound desc = could not find container \"ad805f92fbd3eaf50f40accf7fc8f07da7deb9e86875ff5d197dd8ec96a5072b\": container with ID starting with ad805f92fbd3eaf50f40accf7fc8f07da7deb9e86875ff5d197dd8ec96a5072b not found: ID does not exist"
Oct 31 03:29:09.997679 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:09.997648    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cb458f74f-7bjpx"]
Oct 31 03:29:10.045168 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:10.045097    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5cb458f74f-7bjpx"]
Oct 31 03:29:11.536483 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:11.536446    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="260390fb-c26d-4989-9e8a-a104c398e1c6" path="/var/lib/kubelet/pods/260390fb-c26d-4989-9e8a-a104c398e1c6/volumes"
Oct 31 03:29:31.476892 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.476839    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn" podUID="05c25677-4b67-4a79-bbca-25bcc3ef43f1" containerName="registry" containerID="cri-o://56aa3f2123e2e8ac2976d2279563ddbfc14a45445853e0ed9bb9c3e81f4800da" gracePeriod=30
Oct 31 03:29:31.731324 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.731255    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:29:31.745719 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.745684    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-trusted-ca\") pod \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") "
Oct 31 03:29:31.745847 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.745746    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-kube-api-access-scqrb\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-kube-api-access-scqrb\") pod \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") "
Oct 31 03:29:31.745847 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.745789    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") pod \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") "
Oct 31 03:29:31.745847 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.745830    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-installation-pull-secrets\") pod \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") "
Oct 31 03:29:31.746001 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.745863    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-certificates\") pod \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") "
Oct 31 03:29:31.746001 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.745910    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/05c25677-4b67-4a79-bbca-25bcc3ef43f1-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05c25677-4b67-4a79-bbca-25bcc3ef43f1-ca-trust-extracted\") pod \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") "
Oct 31 03:29:31.746001 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.745970    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-image-registry-private-configuration\") pod \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") "
Oct 31 03:29:31.746142 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.746008    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-bound-sa-token\") pod \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\" (UID: \"05c25677-4b67-4a79-bbca-25bcc3ef43f1\") "
Oct 31 03:29:31.746286 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.746146    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-trusted-ca" pod "05c25677-4b67-4a79-bbca-25bcc3ef43f1" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:29:31.746566 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.746514    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-certificates" pod "05c25677-4b67-4a79-bbca-25bcc3ef43f1" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:29:31.749883 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.749830    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-kube-api-access-scqrb" pod "05c25677-4b67-4a79-bbca-25bcc3ef43f1" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1"). InnerVolumeSpecName "kube-api-access-scqrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:29:31.750107 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.750069    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-bound-sa-token" pod "05c25677-4b67-4a79-bbca-25bcc3ef43f1" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:29:31.750107 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.750091    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-installation-pull-secrets" pod "05c25677-4b67-4a79-bbca-25bcc3ef43f1" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:29:31.750279 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.750104    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls" pod "05c25677-4b67-4a79-bbca-25bcc3ef43f1" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:29:31.751351 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.751326    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-image-registry-private-configuration" pod "05c25677-4b67-4a79-bbca-25bcc3ef43f1" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:29:31.753157 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.753123    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05c25677-4b67-4a79-bbca-25bcc3ef43f1-ca-trust-extracted" pod "05c25677-4b67-4a79-bbca-25bcc3ef43f1" (UID: "05c25677-4b67-4a79-bbca-25bcc3ef43f1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:29:31.847181 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.847157    4926 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-installation-pull-secrets\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:31.847181 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.847178    4926 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-certificates\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:31.847384 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.847190    4926 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/05c25677-4b67-4a79-bbca-25bcc3ef43f1-ca-trust-extracted\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:31.847384 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.847200    4926 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/05c25677-4b67-4a79-bbca-25bcc3ef43f1-image-registry-private-configuration\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:31.847384 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.847209    4926 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-bound-sa-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:31.847384 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.847218    4926 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05c25677-4b67-4a79-bbca-25bcc3ef43f1-trusted-ca\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:31.847384 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.847227    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scqrb\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-kube-api-access-scqrb\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:31.847384 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.847236    4926 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/05c25677-4b67-4a79-bbca-25bcc3ef43f1-registry-tls\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:29:31.873936 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.873908    4926 generic.go:356] "Generic (PLEG): container finished" podID="05c25677-4b67-4a79-bbca-25bcc3ef43f1" containerID="56aa3f2123e2e8ac2976d2279563ddbfc14a45445853e0ed9bb9c3e81f4800da" exitCode=0
Oct 31 03:29:31.874074 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.873956    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn"
Oct 31 03:29:31.874074 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.873961    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn" event={"ID":"05c25677-4b67-4a79-bbca-25bcc3ef43f1","Type":"ContainerDied","Data":"56aa3f2123e2e8ac2976d2279563ddbfc14a45445853e0ed9bb9c3e81f4800da"}
Oct 31 03:29:31.874074 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.874039    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bd6c7ddd-l84xn" event={"ID":"05c25677-4b67-4a79-bbca-25bcc3ef43f1","Type":"ContainerDied","Data":"38b42fd4d1ff5cea3fc9ebb176aa92ed8a0dd4bc8065a42fe7abd36fbdef2682"}
Oct 31 03:29:31.874074 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.874066    4926 scope.go:122] "RemoveContainer" containerID="56aa3f2123e2e8ac2976d2279563ddbfc14a45445853e0ed9bb9c3e81f4800da"
Oct 31 03:29:31.882376 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.882354    4926 scope.go:122] "RemoveContainer" containerID="56aa3f2123e2e8ac2976d2279563ddbfc14a45445853e0ed9bb9c3e81f4800da"
Oct 31 03:29:31.882641 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:29:31.882621    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56aa3f2123e2e8ac2976d2279563ddbfc14a45445853e0ed9bb9c3e81f4800da\": container with ID starting with 56aa3f2123e2e8ac2976d2279563ddbfc14a45445853e0ed9bb9c3e81f4800da not found: ID does not exist" containerID="56aa3f2123e2e8ac2976d2279563ddbfc14a45445853e0ed9bb9c3e81f4800da"
Oct 31 03:29:31.882699 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:31.882650    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56aa3f2123e2e8ac2976d2279563ddbfc14a45445853e0ed9bb9c3e81f4800da"} err="failed to get container status \"56aa3f2123e2e8ac2976d2279563ddbfc14a45445853e0ed9bb9c3e81f4800da\": rpc error: code = NotFound desc = could not find container \"56aa3f2123e2e8ac2976d2279563ddbfc14a45445853e0ed9bb9c3e81f4800da\": container with ID starting with 56aa3f2123e2e8ac2976d2279563ddbfc14a45445853e0ed9bb9c3e81f4800da not found: ID does not exist"
Oct 31 03:29:32.000303 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:32.000200    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8bd6c7ddd-l84xn"]
Oct 31 03:29:32.007136 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:32.007105    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8bd6c7ddd-l84xn"]
Oct 31 03:29:33.537346 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:33.537315    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="05c25677-4b67-4a79-bbca-25bcc3ef43f1" path="/var/lib/kubelet/pods/05c25677-4b67-4a79-bbca-25bcc3ef43f1/volumes"
Oct 31 03:29:35.312006 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:29:35.311975    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:29:44.922571 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:44.922539    4926 generic.go:356] "Generic (PLEG): container finished" podID="df73b930-8122-434b-b2a0-cc3dd2c44d5c" containerID="1f77b08ba82cb7a1aec40762285db923b9970cf488b40e6e2976edd63df9e39d" exitCode=0
Oct 31 03:29:44.923023 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:44.922617    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-s42wk" event={"ID":"df73b930-8122-434b-b2a0-cc3dd2c44d5c","Type":"ContainerDied","Data":"1f77b08ba82cb7a1aec40762285db923b9970cf488b40e6e2976edd63df9e39d"}
Oct 31 03:29:44.923023 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:44.922970    4926 scope.go:122] "RemoveContainer" containerID="1f77b08ba82cb7a1aec40762285db923b9970cf488b40e6e2976edd63df9e39d"
Oct 31 03:29:45.927358 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:45.927324    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-s42wk" event={"ID":"df73b930-8122-434b-b2a0-cc3dd2c44d5c","Type":"ContainerStarted","Data":"894a7558d0e04617aeab87c766f254970684222145f77c42b250682dfff0f648"}
Oct 31 03:29:50.945933 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:50.945849    4926 generic.go:356] "Generic (PLEG): container finished" podID="0bf0c577-0537-46fd-950a-ce5f565a51da" containerID="6cd70920bbea0d3c4edad13fa6154c908830964231e769b92b1e59d67b9ecb9f" exitCode=0
Oct 31 03:29:50.945933 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:50.945908    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n" event={"ID":"0bf0c577-0537-46fd-950a-ce5f565a51da","Type":"ContainerDied","Data":"6cd70920bbea0d3c4edad13fa6154c908830964231e769b92b1e59d67b9ecb9f"}
Oct 31 03:29:50.946386 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:50.946224    4926 scope.go:122] "RemoveContainer" containerID="6cd70920bbea0d3c4edad13fa6154c908830964231e769b92b1e59d67b9ecb9f"
Oct 31 03:29:51.951153 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:51.951111    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nhj2n" event={"ID":"0bf0c577-0537-46fd-950a-ce5f565a51da","Type":"ContainerStarted","Data":"7a2d1a8a75ae01300a11664b04ec5fca55ef0e61eea78d0644ac90576eb8360f"}
Oct 31 03:29:55.965564 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:55.965530    4926 generic.go:356] "Generic (PLEG): container finished" podID="7cc09f0c-627f-4445-b172-9556cab2bbdd" containerID="9621d3494d28955f9d184c4f8ed64231ff1d8c03df7fb4370b1f047947704b16" exitCode=0
Oct 31 03:29:55.966004 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:55.965588    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg" event={"ID":"7cc09f0c-627f-4445-b172-9556cab2bbdd","Type":"ContainerDied","Data":"9621d3494d28955f9d184c4f8ed64231ff1d8c03df7fb4370b1f047947704b16"}
Oct 31 03:29:55.966004 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:55.965863    4926 scope.go:122] "RemoveContainer" containerID="9621d3494d28955f9d184c4f8ed64231ff1d8c03df7fb4370b1f047947704b16"
Oct 31 03:29:56.970488 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:29:56.970457    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-4r9rg" event={"ID":"7cc09f0c-627f-4445-b172-9556cab2bbdd","Type":"ContainerStarted","Data":"7ed65ea5e60790adfe06de4627fbb6109d518fec8ed337bf986a3ee23528e65d"}
Oct 31 03:30:05.311914 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:30:05.311877    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:30:35.311600 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:30:35.311564    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:31:05.312789 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:31:05.312752    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:31:35.312082 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:31:35.312045    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:32:05.311361 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:32:05.311258    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:32:35.312730 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:32:35.312692    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:33:05.311647 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:33:05.311602    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:33:23.539358 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:33:23.539328    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 03:33:23.540783 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:33:23.540749    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 03:33:35.310969 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:33:35.310928    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:34:05.311765 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:34:05.311727    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:34:18.926837 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:18.926804    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"]
Oct 31 03:34:18.927206 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:18.927074    4926 cpu_manager.go:414] "containerMap: removing container" podUID="05c25677-4b67-4a79-bbca-25bcc3ef43f1" containerName="registry"
Oct 31 03:34:18.927206 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:18.927083    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="05c25677-4b67-4a79-bbca-25bcc3ef43f1" containerName="registry"
Oct 31 03:34:18.927206 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:18.927103    4926 cpu_manager.go:414] "containerMap: removing container" podUID="260390fb-c26d-4989-9e8a-a104c398e1c6" containerName="registry"
Oct 31 03:34:18.927206 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:18.927107    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="260390fb-c26d-4989-9e8a-a104c398e1c6" containerName="registry"
Oct 31 03:34:18.927206 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:18.927162    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="05c25677-4b67-4a79-bbca-25bcc3ef43f1" containerName="registry"
Oct 31 03:34:18.927206 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:18.927174    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="260390fb-c26d-4989-9e8a-a104c398e1c6" containerName="registry"
Oct 31 03:34:18.930067 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:18.930051    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:18.981474 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:18.981449    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 31 03:34:18.986633 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:18.986605    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 31 03:34:18.986633 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:18.986607    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 31 03:34:19.028652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.028621    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 31 03:34:19.028866 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.028848    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bfz2c\""
Oct 31 03:34:19.029840 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.029821    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"]
Oct 31 03:34:19.081886 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.081858    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.082023 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.081889    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.082023 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.081911    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.082139 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.082060    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgw9x\" (UniqueName: \"kubernetes.io/projected/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-kube-api-access-dgw9x\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.082139 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.082109    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.082228 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.082181    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.082228 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.082199    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.182760 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.182686    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.182760 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.182738    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgw9x\" (UniqueName: \"kubernetes.io/projected/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-kube-api-access-dgw9x\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.182760 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.182762    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.183007 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.182789    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.183007 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.182806    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.183007 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.182842    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.183007 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.182872    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.183379 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.183346    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.185431 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.185405    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.185533 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.185414    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.185533 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.185511    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.185611 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.185560    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.241662 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.241634    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.241662 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.241651    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgw9x\" (UniqueName: \"kubernetes.io/projected/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-kube-api-access-dgw9x\") pod \"istiod-openshift-gateway-7876bfcb74-7qqv8\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.541210 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.541185    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:19.953827 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.953794    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"]
Oct 31 03:34:19.956899 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:34:19.956873    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b2d2e98_c10d_451a_ad65_179f0ff3bb59.slice/crio-45b12b3f268d2c2929df65c46a0b2e436910f4268fdf8bd6e5508cb287d79acb WatchSource:0}: Error finding container 45b12b3f268d2c2929df65c46a0b2e436910f4268fdf8bd6e5508cb287d79acb: Status 404 returned error can't find the container with id 45b12b3f268d2c2929df65c46a0b2e436910f4268fdf8bd6e5508cb287d79acb
Oct 31 03:34:19.958868 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:19.958852    4926 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 03:34:20.885876 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:20.885843    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8" event={"ID":"1b2d2e98-c10d-451a-ad65-179f0ff3bb59","Type":"ContainerStarted","Data":"45b12b3f268d2c2929df65c46a0b2e436910f4268fdf8bd6e5508cb287d79acb"}
Oct 31 03:34:27.818838 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:27.818807    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:34:27.819140 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:27.818868    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:34:27.913557 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:27.913524    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8" event={"ID":"1b2d2e98-c10d-451a-ad65-179f0ff3bb59","Type":"ContainerStarted","Data":"18a6913261bc2e99fb2b3f71f42e47cfb82d826b0bbcb56d0d193f593b9835c5"}
Oct 31 03:34:27.913733 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:27.913654    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:28.047318 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:28.047248    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8" podStartSLOduration=2.18820408 podStartE2EDuration="10.047233142s" totalImagesPullingTime="7.859029062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:34:28.046399418 +0000 UTC m=+364.631939681" watchObservedRunningTime="2026-10-31 03:34:28.047233142 +0000 UTC m=+364.632773403"
Oct 31 03:34:28.918869 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:28.918840    4926 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-7qqv8 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 03:34:28.919261 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:28.918900    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8" podUID="1b2d2e98-c10d-451a-ad65-179f0ff3bb59" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 03:34:31.918638 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:34:31.918606    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:34:35.312013 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:34:35.311978    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:35:05.311294 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:35:05.311198    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:35:28.951197 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:28.951162    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"]
Oct 31 03:35:28.954536 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:28.954520    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.075922 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.075888    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 31 03:35:29.076141 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.076039    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"]
Oct 31 03:35:29.076141 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.076063    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 31 03:35:29.076141 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.076107    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 31 03:35:29.076329 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.076211    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 31 03:35:29.076395 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.076367    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 31 03:35:29.076449 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.076436    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-lljsf\""
Oct 31 03:35:29.117083 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.117053    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbnm6\" (UniqueName: \"kubernetes.io/projected/4e8dac42-d687-4ce7-ae71-ac564b838557-kube-api-access-vbnm6\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.117254 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.117140    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e8dac42-d687-4ce7-ae71-ac564b838557-tmp\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.117254 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.117177    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e8dac42-d687-4ce7-ae71-ac564b838557-metrics-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.117254 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.117205    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4e8dac42-d687-4ce7-ae71-ac564b838557-manifests\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.117460 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.117294    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4e8dac42-d687-4ce7-ae71-ac564b838557-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.117460 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.117330    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e8dac42-d687-4ce7-ae71-ac564b838557-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.218221 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.218136    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbnm6\" (UniqueName: \"kubernetes.io/projected/4e8dac42-d687-4ce7-ae71-ac564b838557-kube-api-access-vbnm6\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.218221 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.218178    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e8dac42-d687-4ce7-ae71-ac564b838557-tmp\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.218221 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.218198    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e8dac42-d687-4ce7-ae71-ac564b838557-metrics-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.218509 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.218323    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4e8dac42-d687-4ce7-ae71-ac564b838557-manifests\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.218509 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.218407    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4e8dac42-d687-4ce7-ae71-ac564b838557-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.218509 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.218446    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e8dac42-d687-4ce7-ae71-ac564b838557-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.218744 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.218724    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4e8dac42-d687-4ce7-ae71-ac564b838557-manifests\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.220989 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.220961    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e8dac42-d687-4ce7-ae71-ac564b838557-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.220989 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.220979    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4e8dac42-d687-4ce7-ae71-ac564b838557-tmp\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.221135 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.220971    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4e8dac42-d687-4ce7-ae71-ac564b838557-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.221175 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.221158    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e8dac42-d687-4ce7-ae71-ac564b838557-metrics-cert\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.276794 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.276768    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbnm6\" (UniqueName: \"kubernetes.io/projected/4e8dac42-d687-4ce7-ae71-ac564b838557-kube-api-access-vbnm6\") pod \"opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6\" (UID: \"4e8dac42-d687-4ce7-ae71-ac564b838557\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:29.569172 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:29.569091    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:30.100240 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:30.100218    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"]
Oct 31 03:35:30.102504 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:35:30.102480    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e8dac42_d687_4ce7_ae71_ac564b838557.slice/crio-6717aa35997426e4cf264bc0fa8ff992660dcfd104f5405710191669ec8aa6e1 WatchSource:0}: Error finding container 6717aa35997426e4cf264bc0fa8ff992660dcfd104f5405710191669ec8aa6e1: Status 404 returned error can't find the container with id 6717aa35997426e4cf264bc0fa8ff992660dcfd104f5405710191669ec8aa6e1
Oct 31 03:35:30.140229 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:30.140202    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6" event={"ID":"4e8dac42-d687-4ce7-ae71-ac564b838557","Type":"ContainerStarted","Data":"6717aa35997426e4cf264bc0fa8ff992660dcfd104f5405710191669ec8aa6e1"}
Oct 31 03:35:34.161190 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:34.161163    4926 generic.go:356] "Generic (PLEG): container finished" podID="4e8dac42-d687-4ce7-ae71-ac564b838557" containerID="258f07b295c2cc1c73eb61af9eb78184aa7ac6dd5a12f08047dabcbcee0f4bf0" exitCode=0
Oct 31 03:35:34.161595 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:34.161244    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6" event={"ID":"4e8dac42-d687-4ce7-ae71-ac564b838557","Type":"ContainerDied","Data":"258f07b295c2cc1c73eb61af9eb78184aa7ac6dd5a12f08047dabcbcee0f4bf0"}
Oct 31 03:35:35.311624 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:35:35.311589    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:35:36.170863 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:36.170819    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6" event={"ID":"4e8dac42-d687-4ce7-ae71-ac564b838557","Type":"ContainerStarted","Data":"e269b6efff7b32e9a88e2eedd432a8022560b4971e26121e600d8ce2c1be50cc"}
Oct 31 03:35:36.171065 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:36.170936    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:35:36.448199 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:36.448110    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6" podStartSLOduration=3.5250026500000002 podStartE2EDuration="8.448096621s" totalImagesPullingTime="4.461406962s" totalInitContainerRuntime="461.687009ms" isStatefulPod=false podCreationTimestamp="2026-10-31 03:35:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:35:36.400669584 +0000 UTC m=+432.986209849" watchObservedRunningTime="2026-10-31 03:35:36.448096621 +0000 UTC m=+433.033636883"
Oct 31 03:35:47.178325 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:35:47.178295    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-6dcdbf6b7b-m5tg6"
Oct 31 03:36:03.178923 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.178884    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5897747699-kzfcc"]
Oct 31 03:36:03.189581 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.189555    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5897747699-kzfcc"
Oct 31 03:36:03.238542 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.238513    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5897747699-kzfcc"]
Oct 31 03:36:03.284365 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.284330    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 31 03:36:03.284518 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.284389    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mclvc\""
Oct 31 03:36:03.284518 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.284449    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 31 03:36:03.286677 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.286658    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/038905c7-ab91-4a97-b514-7cf197c09a31-tls-certs\") pod \"kube-auth-proxy-5897747699-kzfcc\" (UID: \"038905c7-ab91-4a97-b514-7cf197c09a31\") " pod="openshift-ingress/kube-auth-proxy-5897747699-kzfcc"
Oct 31 03:36:03.286732 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.286696    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/038905c7-ab91-4a97-b514-7cf197c09a31-tmp\") pod \"kube-auth-proxy-5897747699-kzfcc\" (UID: \"038905c7-ab91-4a97-b514-7cf197c09a31\") " pod="openshift-ingress/kube-auth-proxy-5897747699-kzfcc"
Oct 31 03:36:03.286732 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.286722    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf27k\" (UniqueName: \"kubernetes.io/projected/038905c7-ab91-4a97-b514-7cf197c09a31-kube-api-access-xf27k\") pod \"kube-auth-proxy-5897747699-kzfcc\" (UID: \"038905c7-ab91-4a97-b514-7cf197c09a31\") " pod="openshift-ingress/kube-auth-proxy-5897747699-kzfcc"
Oct 31 03:36:03.387910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.387871    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf27k\" (UniqueName: \"kubernetes.io/projected/038905c7-ab91-4a97-b514-7cf197c09a31-kube-api-access-xf27k\") pod \"kube-auth-proxy-5897747699-kzfcc\" (UID: \"038905c7-ab91-4a97-b514-7cf197c09a31\") " pod="openshift-ingress/kube-auth-proxy-5897747699-kzfcc"
Oct 31 03:36:03.388105 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.387938    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/038905c7-ab91-4a97-b514-7cf197c09a31-tls-certs\") pod \"kube-auth-proxy-5897747699-kzfcc\" (UID: \"038905c7-ab91-4a97-b514-7cf197c09a31\") " pod="openshift-ingress/kube-auth-proxy-5897747699-kzfcc"
Oct 31 03:36:03.388105 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.387969    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/038905c7-ab91-4a97-b514-7cf197c09a31-tmp\") pod \"kube-auth-proxy-5897747699-kzfcc\" (UID: \"038905c7-ab91-4a97-b514-7cf197c09a31\") " pod="openshift-ingress/kube-auth-proxy-5897747699-kzfcc"
Oct 31 03:36:03.390366 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.390341    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/038905c7-ab91-4a97-b514-7cf197c09a31-tmp\") pod \"kube-auth-proxy-5897747699-kzfcc\" (UID: \"038905c7-ab91-4a97-b514-7cf197c09a31\") " pod="openshift-ingress/kube-auth-proxy-5897747699-kzfcc"
Oct 31 03:36:03.390690 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.390669    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/038905c7-ab91-4a97-b514-7cf197c09a31-tls-certs\") pod \"kube-auth-proxy-5897747699-kzfcc\" (UID: \"038905c7-ab91-4a97-b514-7cf197c09a31\") " pod="openshift-ingress/kube-auth-proxy-5897747699-kzfcc"
Oct 31 03:36:03.446970 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.446894    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf27k\" (UniqueName: \"kubernetes.io/projected/038905c7-ab91-4a97-b514-7cf197c09a31-kube-api-access-xf27k\") pod \"kube-auth-proxy-5897747699-kzfcc\" (UID: \"038905c7-ab91-4a97-b514-7cf197c09a31\") " pod="openshift-ingress/kube-auth-proxy-5897747699-kzfcc"
Oct 31 03:36:03.501060 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.501030    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5897747699-kzfcc"
Oct 31 03:36:03.872855 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:03.872787    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5897747699-kzfcc"]
Oct 31 03:36:03.875145 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:36:03.875115    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod038905c7_ab91_4a97_b514_7cf197c09a31.slice/crio-c75d5809a09780ed09c737ebe65f0cc9f1800797ab768d45b1c383f9d793e737 WatchSource:0}: Error finding container c75d5809a09780ed09c737ebe65f0cc9f1800797ab768d45b1c383f9d793e737: Status 404 returned error can't find the container with id c75d5809a09780ed09c737ebe65f0cc9f1800797ab768d45b1c383f9d793e737
Oct 31 03:36:04.283383 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:04.283349    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5897747699-kzfcc" event={"ID":"038905c7-ab91-4a97-b514-7cf197c09a31","Type":"ContainerStarted","Data":"c75d5809a09780ed09c737ebe65f0cc9f1800797ab768d45b1c383f9d793e737"}
Oct 31 03:36:05.318364 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:36:05.318300    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:36:07.297613 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:07.297515    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5897747699-kzfcc" event={"ID":"038905c7-ab91-4a97-b514-7cf197c09a31","Type":"ContainerStarted","Data":"a3ef8b7ac934897cbfc89583a1067163c58c38cf6cda912f0ff7587e1ce8522c"}
Oct 31 03:36:07.412550 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:07.412498    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5897747699-kzfcc" podStartSLOduration=1.279052159 podStartE2EDuration="4.412483245s" totalImagesPullingTime="3.133431086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:36:07.411708834 +0000 UTC m=+463.997249096" watchObservedRunningTime="2026-10-31 03:36:07.412483245 +0000 UTC m=+463.998023507"
Oct 31 03:36:16.971299 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:16.971239    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"]
Oct 31 03:36:16.975873 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:16.975849    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:36:17.074965 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.074926    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\""
Oct 31 03:36:17.074965 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.074955    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\""
Oct 31 03:36:17.075170 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.074937    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\""
Oct 31 03:36:17.075170 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.074963    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Oct 31 03:36:17.075170 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.075008    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Oct 31 03:36:17.075170 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.075137    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-j92jb\""
Oct 31 03:36:17.076300 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.076283    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"]
Oct 31 03:36:17.111772 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.111741    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxzcd\" (UniqueName: \"kubernetes.io/projected/29153ea2-6dc6-40ad-b797-bac950be7513-kube-api-access-fxzcd\") pod \"jobset-controller-manager-645db6d798-8g6bb\" (UID: \"29153ea2-6dc6-40ad-b797-bac950be7513\") " pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:36:17.111772 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.111773    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29153ea2-6dc6-40ad-b797-bac950be7513-metrics-certs\") pod \"jobset-controller-manager-645db6d798-8g6bb\" (UID: \"29153ea2-6dc6-40ad-b797-bac950be7513\") " pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:36:17.111974 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.111805    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/29153ea2-6dc6-40ad-b797-bac950be7513-manager-config\") pod \"jobset-controller-manager-645db6d798-8g6bb\" (UID: \"29153ea2-6dc6-40ad-b797-bac950be7513\") " pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:36:17.111974 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.111927    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29153ea2-6dc6-40ad-b797-bac950be7513-cert\") pod \"jobset-controller-manager-645db6d798-8g6bb\" (UID: \"29153ea2-6dc6-40ad-b797-bac950be7513\") " pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:36:17.212727 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.212691    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/29153ea2-6dc6-40ad-b797-bac950be7513-manager-config\") pod \"jobset-controller-manager-645db6d798-8g6bb\" (UID: \"29153ea2-6dc6-40ad-b797-bac950be7513\") " pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:36:17.212921 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.212737    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29153ea2-6dc6-40ad-b797-bac950be7513-cert\") pod \"jobset-controller-manager-645db6d798-8g6bb\" (UID: \"29153ea2-6dc6-40ad-b797-bac950be7513\") " pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:36:17.212921 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.212779    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxzcd\" (UniqueName: \"kubernetes.io/projected/29153ea2-6dc6-40ad-b797-bac950be7513-kube-api-access-fxzcd\") pod \"jobset-controller-manager-645db6d798-8g6bb\" (UID: \"29153ea2-6dc6-40ad-b797-bac950be7513\") " pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:36:17.212921 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.212799    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29153ea2-6dc6-40ad-b797-bac950be7513-metrics-certs\") pod \"jobset-controller-manager-645db6d798-8g6bb\" (UID: \"29153ea2-6dc6-40ad-b797-bac950be7513\") " pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:36:17.213410 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.213384    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/29153ea2-6dc6-40ad-b797-bac950be7513-manager-config\") pod \"jobset-controller-manager-645db6d798-8g6bb\" (UID: \"29153ea2-6dc6-40ad-b797-bac950be7513\") " pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:36:17.215320 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.215300    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29153ea2-6dc6-40ad-b797-bac950be7513-metrics-certs\") pod \"jobset-controller-manager-645db6d798-8g6bb\" (UID: \"29153ea2-6dc6-40ad-b797-bac950be7513\") " pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:36:17.215404 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.215334    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29153ea2-6dc6-40ad-b797-bac950be7513-cert\") pod \"jobset-controller-manager-645db6d798-8g6bb\" (UID: \"29153ea2-6dc6-40ad-b797-bac950be7513\") " pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:36:17.273793 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.273729    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxzcd\" (UniqueName: \"kubernetes.io/projected/29153ea2-6dc6-40ad-b797-bac950be7513-kube-api-access-fxzcd\") pod \"jobset-controller-manager-645db6d798-8g6bb\" (UID: \"29153ea2-6dc6-40ad-b797-bac950be7513\") " pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:36:17.287600 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.287572    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:36:17.650859 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:17.650736    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"]
Oct 31 03:36:17.653089 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:36:17.653060    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29153ea2_6dc6_40ad_b797_bac950be7513.slice/crio-8ede85b687a3434235e9f8e08d4e7ee41cda39f1644c1838527b32fd1511d3ff WatchSource:0}: Error finding container 8ede85b687a3434235e9f8e08d4e7ee41cda39f1644c1838527b32fd1511d3ff: Status 404 returned error can't find the container with id 8ede85b687a3434235e9f8e08d4e7ee41cda39f1644c1838527b32fd1511d3ff
Oct 31 03:36:18.343667 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:18.343638    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb" event={"ID":"29153ea2-6dc6-40ad-b797-bac950be7513","Type":"ContainerStarted","Data":"8ede85b687a3434235e9f8e08d4e7ee41cda39f1644c1838527b32fd1511d3ff"}
Oct 31 03:36:24.369607 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:24.369566    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb" event={"ID":"29153ea2-6dc6-40ad-b797-bac950be7513","Type":"ContainerStarted","Data":"5f1df51431e562fd4823978c8c7d1c5c7effd42acb16b132a98c00a1b4674a1f"}
Oct 31 03:36:24.369997 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:24.369710    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:36:24.519034 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:24.518980    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb" podStartSLOduration=2.522153923 podStartE2EDuration="8.518964691s" totalImagesPullingTime="5.996810768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:36:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:36:24.517890903 +0000 UTC m=+481.103431166" watchObservedRunningTime="2026-10-31 03:36:24.518964691 +0000 UTC m=+481.104504953"
Oct 31 03:36:35.311836 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:36:35.311802    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:36:35.379033 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:36:35.379008    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-645db6d798-8g6bb"
Oct 31 03:37:05.312240 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:37:05.312202    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:37:35.312552 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:37:35.312506    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:37:39.741123 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:39.741096    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"]
Oct 31 03:37:39.741553 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:39.741286    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8" podUID="1b2d2e98-c10d-451a-ad65-179f0ff3bb59" containerName="discovery" containerID="cri-o://18a6913261bc2e99fb2b3f71f42e47cfb82d826b0bbcb56d0d193f593b9835c5" gracePeriod=30
Oct 31 03:37:39.985624 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:39.985600    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:37:39.997029 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:39.996973    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-local-certs\") pod \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") "
Oct 31 03:37:39.997029 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:39.997008    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-cacerts\" (UniqueName: \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-cacerts\") pod \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") "
Oct 31 03:37:39.997213 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:39.997048    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-csr-ca-configmap\") pod \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") "
Oct 31 03:37:39.997213 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:39.997065    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-csr-dns-cert\") pod \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") "
Oct 31 03:37:39.997213 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:39.997085    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-token\" (UniqueName: \"kubernetes.io/projected/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-token\") pod \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") "
Oct 31 03:37:39.997213 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:39.997120    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-kube-api-access-dgw9x\" (UniqueName: \"kubernetes.io/projected/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-kube-api-access-dgw9x\") pod \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") "
Oct 31 03:37:39.997213 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:39.997164    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-kubeconfig\") pod \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\" (UID: \"1b2d2e98-c10d-451a-ad65-179f0ff3bb59\") "
Oct 31 03:37:39.997543 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:39.997518    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-csr-ca-configmap" pod "1b2d2e98-c10d-451a-ad65-179f0ff3bb59" (UID: "1b2d2e98-c10d-451a-ad65-179f0ff3bb59"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:37:39.999484 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:39.999459    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-local-certs" pod "1b2d2e98-c10d-451a-ad65-179f0ff3bb59" (UID: "1b2d2e98-c10d-451a-ad65-179f0ff3bb59"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:37:39.999722 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:39.999690    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-kube-api-access-dgw9x" pod "1b2d2e98-c10d-451a-ad65-179f0ff3bb59" (UID: "1b2d2e98-c10d-451a-ad65-179f0ff3bb59"). InnerVolumeSpecName "kube-api-access-dgw9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:37:39.999873 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:39.999858    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-cacerts" pod "1b2d2e98-c10d-451a-ad65-179f0ff3bb59" (UID: "1b2d2e98-c10d-451a-ad65-179f0ff3bb59"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:37:39.999937 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:39.999904    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-csr-dns-cert" pod "1b2d2e98-c10d-451a-ad65-179f0ff3bb59" (UID: "1b2d2e98-c10d-451a-ad65-179f0ff3bb59"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:37:40.000051 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.000035    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-kubeconfig" pod "1b2d2e98-c10d-451a-ad65-179f0ff3bb59" (UID: "1b2d2e98-c10d-451a-ad65-179f0ff3bb59"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:37:40.000088 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.000049    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-token" pod "1b2d2e98-c10d-451a-ad65-179f0ff3bb59" (UID: "1b2d2e98-c10d-451a-ad65-179f0ff3bb59"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:37:40.097858 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.097823    4926 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-csr-ca-configmap\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:37:40.097858 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.097852    4926 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-csr-dns-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:37:40.098046 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.097867    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:37:40.098046 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.097880    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgw9x\" (UniqueName: \"kubernetes.io/projected/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-kube-api-access-dgw9x\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:37:40.098046 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.097892    4926 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-istio-kubeconfig\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:37:40.098046 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.097903    4926 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-local-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:37:40.098046 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.097914    4926 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1b2d2e98-c10d-451a-ad65-179f0ff3bb59-cacerts\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:37:40.665562 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.665530    4926 generic.go:356] "Generic (PLEG): container finished" podID="1b2d2e98-c10d-451a-ad65-179f0ff3bb59" containerID="18a6913261bc2e99fb2b3f71f42e47cfb82d826b0bbcb56d0d193f593b9835c5" exitCode=0
Oct 31 03:37:40.665742 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.665589    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8" event={"ID":"1b2d2e98-c10d-451a-ad65-179f0ff3bb59","Type":"ContainerDied","Data":"18a6913261bc2e99fb2b3f71f42e47cfb82d826b0bbcb56d0d193f593b9835c5"}
Oct 31 03:37:40.665742 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.665591    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"
Oct 31 03:37:40.665742 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.665627    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8" event={"ID":"1b2d2e98-c10d-451a-ad65-179f0ff3bb59","Type":"ContainerDied","Data":"45b12b3f268d2c2929df65c46a0b2e436910f4268fdf8bd6e5508cb287d79acb"}
Oct 31 03:37:40.665742 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.665647    4926 scope.go:122] "RemoveContainer" containerID="18a6913261bc2e99fb2b3f71f42e47cfb82d826b0bbcb56d0d193f593b9835c5"
Oct 31 03:37:40.674754 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.674735    4926 scope.go:122] "RemoveContainer" containerID="18a6913261bc2e99fb2b3f71f42e47cfb82d826b0bbcb56d0d193f593b9835c5"
Oct 31 03:37:40.674958 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:37:40.674942    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18a6913261bc2e99fb2b3f71f42e47cfb82d826b0bbcb56d0d193f593b9835c5\": container with ID starting with 18a6913261bc2e99fb2b3f71f42e47cfb82d826b0bbcb56d0d193f593b9835c5 not found: ID does not exist" containerID="18a6913261bc2e99fb2b3f71f42e47cfb82d826b0bbcb56d0d193f593b9835c5"
Oct 31 03:37:40.675008 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.674966    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18a6913261bc2e99fb2b3f71f42e47cfb82d826b0bbcb56d0d193f593b9835c5"} err="failed to get container status \"18a6913261bc2e99fb2b3f71f42e47cfb82d826b0bbcb56d0d193f593b9835c5\": rpc error: code = NotFound desc = could not find container \"18a6913261bc2e99fb2b3f71f42e47cfb82d826b0bbcb56d0d193f593b9835c5\": container with ID starting with 18a6913261bc2e99fb2b3f71f42e47cfb82d826b0bbcb56d0d193f593b9835c5 not found: ID does not exist"
Oct 31 03:37:40.873011 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.872974    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"]
Oct 31 03:37:40.920548 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:40.920475    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-7qqv8"]
Oct 31 03:37:41.536501 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:41.536469    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b2d2e98-c10d-451a-ad65-179f0ff3bb59" path="/var/lib/kubelet/pods/1b2d2e98-c10d-451a-ad65-179f0ff3bb59/volumes"
Oct 31 03:37:45.576255 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.576223    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"]
Oct 31 03:37:45.576639 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.576529    4926 cpu_manager.go:414] "containerMap: removing container" podUID="1b2d2e98-c10d-451a-ad65-179f0ff3bb59" containerName="discovery"
Oct 31 03:37:45.576639 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.576539    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b2d2e98-c10d-451a-ad65-179f0ff3bb59" containerName="discovery"
Oct 31 03:37:45.576639 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.576602    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b2d2e98-c10d-451a-ad65-179f0ff3bb59" containerName="discovery"
Oct 31 03:37:45.580641 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.580618    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:37:45.641333 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.641289    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc8db575-6fbe-44dc-8383-c520acde9cf9-metrics-certs\") pod \"maas-api-6d669c7fcd-xzzl6\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:37:45.641487 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.641354    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkqfl\" (UniqueName: \"kubernetes.io/projected/fc8db575-6fbe-44dc-8383-c520acde9cf9-kube-api-access-vkqfl\") pod \"maas-api-6d669c7fcd-xzzl6\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:37:45.641487 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.641411    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc8db575-6fbe-44dc-8383-c520acde9cf9-maas-api-tls\") pod \"maas-api-6d669c7fcd-xzzl6\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:37:45.641603 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.641497    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc8db575-6fbe-44dc-8383-c520acde9cf9-openshift-service-ca\") pod \"maas-api-6d669c7fcd-xzzl6\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:37:45.678205 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.678182    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 31 03:37:45.678584 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.678552    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 31 03:37:45.678584 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.678571    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 31 03:37:45.678779 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.678667    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gc447\""
Oct 31 03:37:45.678779 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.678709    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 31 03:37:45.679891 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.679871    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"]
Oct 31 03:37:45.741902 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.741875    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc8db575-6fbe-44dc-8383-c520acde9cf9-metrics-certs\") pod \"maas-api-6d669c7fcd-xzzl6\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:37:45.742034 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.741907    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkqfl\" (UniqueName: \"kubernetes.io/projected/fc8db575-6fbe-44dc-8383-c520acde9cf9-kube-api-access-vkqfl\") pod \"maas-api-6d669c7fcd-xzzl6\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:37:45.742034 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.741941    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc8db575-6fbe-44dc-8383-c520acde9cf9-maas-api-tls\") pod \"maas-api-6d669c7fcd-xzzl6\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:37:45.742034 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.741996    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc8db575-6fbe-44dc-8383-c520acde9cf9-openshift-service-ca\") pod \"maas-api-6d669c7fcd-xzzl6\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:37:45.742798 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.742777    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc8db575-6fbe-44dc-8383-c520acde9cf9-openshift-service-ca\") pod \"maas-api-6d669c7fcd-xzzl6\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:37:45.744495 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.744475    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc8db575-6fbe-44dc-8383-c520acde9cf9-maas-api-tls\") pod \"maas-api-6d669c7fcd-xzzl6\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:37:45.744576 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.744482    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc8db575-6fbe-44dc-8383-c520acde9cf9-metrics-certs\") pod \"maas-api-6d669c7fcd-xzzl6\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:37:45.801027 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.800980    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkqfl\" (UniqueName: \"kubernetes.io/projected/fc8db575-6fbe-44dc-8383-c520acde9cf9-kube-api-access-vkqfl\") pod \"maas-api-6d669c7fcd-xzzl6\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:37:45.893370 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:45.893261    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:37:46.258085 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:46.258054    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"]
Oct 31 03:37:46.259839 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:37:46.259812    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc8db575_6fbe_44dc_8383_c520acde9cf9.slice/crio-61b863051727673df1ac8d347e76e6a83cbe8b785c3ac4dd038e523010621904 WatchSource:0}: Error finding container 61b863051727673df1ac8d347e76e6a83cbe8b785c3ac4dd038e523010621904: Status 404 returned error can't find the container with id 61b863051727673df1ac8d347e76e6a83cbe8b785c3ac4dd038e523010621904
Oct 31 03:37:46.690899 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:46.690812    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6" event={"ID":"fc8db575-6fbe-44dc-8383-c520acde9cf9","Type":"ContainerStarted","Data":"61b863051727673df1ac8d347e76e6a83cbe8b785c3ac4dd038e523010621904"}
Oct 31 03:37:49.705281 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:49.705227    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6" event={"ID":"fc8db575-6fbe-44dc-8383-c520acde9cf9","Type":"ContainerStarted","Data":"ae53650b3425982a537eb35ad4a2f68898ac414c13b4a67db5600b751f8b7d50"}
Oct 31 03:37:49.705767 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:49.705330    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:37:49.819206 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:49.819157    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6" podStartSLOduration=1.62375975 podStartE2EDuration="4.819141697s" totalImagesPullingTime="3.195381947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:37:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:37:49.818751767 +0000 UTC m=+566.404292028" watchObservedRunningTime="2026-10-31 03:37:49.819141697 +0000 UTC m=+566.404682006"
Oct 31 03:37:55.714815 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:37:55.714787    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:38:05.311940 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:38:05.311839    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:38:16.789993 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:16.789957    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-68c9679795-4c74x"]
Oct 31 03:38:16.793182 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:16.793166    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:16.848580 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:16.848547    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c9679795-4c74x"]
Oct 31 03:38:16.902995 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:16.902964    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e5af999-10c2-4fe8-ae65-800b96a87896-openshift-service-ca\") pod \"maas-api-68c9679795-4c74x\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:16.903156 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:16.903001    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e5af999-10c2-4fe8-ae65-800b96a87896-metrics-certs\") pod \"maas-api-68c9679795-4c74x\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:16.903156 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:16.903037    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-867vq\" (UniqueName: \"kubernetes.io/projected/0e5af999-10c2-4fe8-ae65-800b96a87896-kube-api-access-867vq\") pod \"maas-api-68c9679795-4c74x\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:16.903231 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:16.903197    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e5af999-10c2-4fe8-ae65-800b96a87896-maas-api-tls\") pod \"maas-api-68c9679795-4c74x\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:17.004514 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:17.004485    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-867vq\" (UniqueName: \"kubernetes.io/projected/0e5af999-10c2-4fe8-ae65-800b96a87896-kube-api-access-867vq\") pod \"maas-api-68c9679795-4c74x\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:17.004678 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:17.004537    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e5af999-10c2-4fe8-ae65-800b96a87896-maas-api-tls\") pod \"maas-api-68c9679795-4c74x\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:17.004678 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:17.004656    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e5af999-10c2-4fe8-ae65-800b96a87896-openshift-service-ca\") pod \"maas-api-68c9679795-4c74x\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:17.004749 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:17.004693    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e5af999-10c2-4fe8-ae65-800b96a87896-metrics-certs\") pod \"maas-api-68c9679795-4c74x\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:17.005366 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:17.005343    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e5af999-10c2-4fe8-ae65-800b96a87896-openshift-service-ca\") pod \"maas-api-68c9679795-4c74x\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:17.007144 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:17.007122    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e5af999-10c2-4fe8-ae65-800b96a87896-metrics-certs\") pod \"maas-api-68c9679795-4c74x\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:17.007241 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:17.007164    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e5af999-10c2-4fe8-ae65-800b96a87896-maas-api-tls\") pod \"maas-api-68c9679795-4c74x\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:17.066768 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:17.066701    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-867vq\" (UniqueName: \"kubernetes.io/projected/0e5af999-10c2-4fe8-ae65-800b96a87896-kube-api-access-867vq\") pod \"maas-api-68c9679795-4c74x\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") " pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:17.105607 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:17.105574    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:17.475518 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:38:17.475487    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e5af999_10c2_4fe8_ae65_800b96a87896.slice/crio-764652539bfa6052b3f93ebebe0bf58a20f06ac198d2698ede220ccc203df77b WatchSource:0}: Error finding container 764652539bfa6052b3f93ebebe0bf58a20f06ac198d2698ede220ccc203df77b: Status 404 returned error can't find the container with id 764652539bfa6052b3f93ebebe0bf58a20f06ac198d2698ede220ccc203df77b
Oct 31 03:38:17.480900 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:17.478219    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c9679795-4c74x"]
Oct 31 03:38:17.813614 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:17.813530    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x" event={"ID":"0e5af999-10c2-4fe8-ae65-800b96a87896","Type":"ContainerStarted","Data":"764652539bfa6052b3f93ebebe0bf58a20f06ac198d2698ede220ccc203df77b"}
Oct 31 03:38:18.818781 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:18.818743    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x" event={"ID":"0e5af999-10c2-4fe8-ae65-800b96a87896","Type":"ContainerStarted","Data":"198cdb9e51bf0e45a9dfb3f981adc778383fe3c62ce407e4c38825196121add1"}
Oct 31 03:38:18.819207 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:18.818803    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:18.938991 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:18.938936    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x" podStartSLOduration=2.048737473 podStartE2EDuration="2.938917546s" totalImagesPullingTime="890.180073ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:38:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:38:18.938262496 +0000 UTC m=+595.523802758" watchObservedRunningTime="2026-10-31 03:38:18.938917546 +0000 UTC m=+595.524457809"
Oct 31 03:38:23.572701 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:23.572675    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 03:38:23.573039 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:23.573024    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 03:38:24.828026 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:24.827995    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 03:38:25.000861 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.000822    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"]
Oct 31 03:38:25.001070 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.001046    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6" podUID="fc8db575-6fbe-44dc-8383-c520acde9cf9" containerName="maas-api" containerID="cri-o://ae53650b3425982a537eb35ad4a2f68898ac414c13b4a67db5600b751f8b7d50" gracePeriod=30
Oct 31 03:38:25.243075 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.243051    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:38:25.374083 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.373999    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc8db575-6fbe-44dc-8383-c520acde9cf9-kube-api-access-vkqfl\" (UniqueName: \"kubernetes.io/projected/fc8db575-6fbe-44dc-8383-c520acde9cf9-kube-api-access-vkqfl\") pod \"fc8db575-6fbe-44dc-8383-c520acde9cf9\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") "
Oct 31 03:38:25.374083 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.374057    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc8db575-6fbe-44dc-8383-c520acde9cf9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc8db575-6fbe-44dc-8383-c520acde9cf9-maas-api-tls\") pod \"fc8db575-6fbe-44dc-8383-c520acde9cf9\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") "
Oct 31 03:38:25.374356 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.374092    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc8db575-6fbe-44dc-8383-c520acde9cf9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc8db575-6fbe-44dc-8383-c520acde9cf9-openshift-service-ca\") pod \"fc8db575-6fbe-44dc-8383-c520acde9cf9\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") "
Oct 31 03:38:25.374356 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.374146    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc8db575-6fbe-44dc-8383-c520acde9cf9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc8db575-6fbe-44dc-8383-c520acde9cf9-metrics-certs\") pod \"fc8db575-6fbe-44dc-8383-c520acde9cf9\" (UID: \"fc8db575-6fbe-44dc-8383-c520acde9cf9\") "
Oct 31 03:38:25.374626 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.374577    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc8db575-6fbe-44dc-8383-c520acde9cf9-openshift-service-ca" pod "fc8db575-6fbe-44dc-8383-c520acde9cf9" (UID: "fc8db575-6fbe-44dc-8383-c520acde9cf9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:38:25.376606 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.376576    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc8db575-6fbe-44dc-8383-c520acde9cf9-kube-api-access-vkqfl" pod "fc8db575-6fbe-44dc-8383-c520acde9cf9" (UID: "fc8db575-6fbe-44dc-8383-c520acde9cf9"). InnerVolumeSpecName "kube-api-access-vkqfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:38:25.376707 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.376644    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc8db575-6fbe-44dc-8383-c520acde9cf9-metrics-certs" pod "fc8db575-6fbe-44dc-8383-c520acde9cf9" (UID: "fc8db575-6fbe-44dc-8383-c520acde9cf9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:38:25.376707 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.376685    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc8db575-6fbe-44dc-8383-c520acde9cf9-maas-api-tls" pod "fc8db575-6fbe-44dc-8383-c520acde9cf9" (UID: "fc8db575-6fbe-44dc-8383-c520acde9cf9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 03:38:25.474858 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.474810    4926 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc8db575-6fbe-44dc-8383-c520acde9cf9-openshift-service-ca\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:38:25.474858 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.474855    4926 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc8db575-6fbe-44dc-8383-c520acde9cf9-metrics-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:38:25.474858 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.474866    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkqfl\" (UniqueName: \"kubernetes.io/projected/fc8db575-6fbe-44dc-8383-c520acde9cf9-kube-api-access-vkqfl\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:38:25.474858 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.474876    4926 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc8db575-6fbe-44dc-8383-c520acde9cf9-maas-api-tls\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:38:25.848352 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.848310    4926 generic.go:356] "Generic (PLEG): container finished" podID="fc8db575-6fbe-44dc-8383-c520acde9cf9" containerID="ae53650b3425982a537eb35ad4a2f68898ac414c13b4a67db5600b751f8b7d50" exitCode=0
Oct 31 03:38:25.848739 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.848359    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"
Oct 31 03:38:25.848739 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.848393    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6" event={"ID":"fc8db575-6fbe-44dc-8383-c520acde9cf9","Type":"ContainerDied","Data":"ae53650b3425982a537eb35ad4a2f68898ac414c13b4a67db5600b751f8b7d50"}
Oct 31 03:38:25.848739 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.848447    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6" event={"ID":"fc8db575-6fbe-44dc-8383-c520acde9cf9","Type":"ContainerDied","Data":"61b863051727673df1ac8d347e76e6a83cbe8b785c3ac4dd038e523010621904"}
Oct 31 03:38:25.848739 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.848470    4926 scope.go:122] "RemoveContainer" containerID="ae53650b3425982a537eb35ad4a2f68898ac414c13b4a67db5600b751f8b7d50"
Oct 31 03:38:25.856986 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.856966    4926 scope.go:122] "RemoveContainer" containerID="ae53650b3425982a537eb35ad4a2f68898ac414c13b4a67db5600b751f8b7d50"
Oct 31 03:38:25.857251 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:38:25.857227    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae53650b3425982a537eb35ad4a2f68898ac414c13b4a67db5600b751f8b7d50\": container with ID starting with ae53650b3425982a537eb35ad4a2f68898ac414c13b4a67db5600b751f8b7d50 not found: ID does not exist" containerID="ae53650b3425982a537eb35ad4a2f68898ac414c13b4a67db5600b751f8b7d50"
Oct 31 03:38:25.857358 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:25.857287    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae53650b3425982a537eb35ad4a2f68898ac414c13b4a67db5600b751f8b7d50"} err="failed to get container status \"ae53650b3425982a537eb35ad4a2f68898ac414c13b4a67db5600b751f8b7d50\": rpc error: code = NotFound desc = could not find container \"ae53650b3425982a537eb35ad4a2f68898ac414c13b4a67db5600b751f8b7d50\": container with ID starting with ae53650b3425982a537eb35ad4a2f68898ac414c13b4a67db5600b751f8b7d50 not found: ID does not exist"
Oct 31 03:38:26.016752 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:26.016710    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"]
Oct 31 03:38:26.068069 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:26.068036    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-xzzl6"]
Oct 31 03:38:27.536253 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:27.536218    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc8db575-6fbe-44dc-8383-c520acde9cf9" path="/var/lib/kubelet/pods/fc8db575-6fbe-44dc-8383-c520acde9cf9/volumes"
Oct 31 03:38:35.311791 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:38:35.311750    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:38:40.561494 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.561457    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt"]
Oct 31 03:38:40.561949 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.561782    4926 cpu_manager.go:414] "containerMap: removing container" podUID="fc8db575-6fbe-44dc-8383-c520acde9cf9" containerName="maas-api"
Oct 31 03:38:40.561949 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.561795    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc8db575-6fbe-44dc-8383-c520acde9cf9" containerName="maas-api"
Oct 31 03:38:40.561949 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.561874    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc8db575-6fbe-44dc-8383-c520acde9cf9" containerName="maas-api"
Oct 31 03:38:40.564814 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.564797    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt"
Oct 31 03:38:40.597620 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.597592    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7386e379-d04b-44b9-8107-23139ac0f6c6-cert\") pod \"llmisvc-controller-manager-6466c554d8-jkdzt\" (UID: \"7386e379-d04b-44b9-8107-23139ac0f6c6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt"
Oct 31 03:38:40.597620 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.597623    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbfqt\" (UniqueName: \"kubernetes.io/projected/7386e379-d04b-44b9-8107-23139ac0f6c6-kube-api-access-dbfqt\") pod \"llmisvc-controller-manager-6466c554d8-jkdzt\" (UID: \"7386e379-d04b-44b9-8107-23139ac0f6c6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt"
Oct 31 03:38:40.619216 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.619184    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt"]
Oct 31 03:38:40.669111 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.669080    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 31 03:38:40.670117 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.670100    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 31 03:38:40.670195 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.670127    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-25d4f\""
Oct 31 03:38:40.670236 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.670186    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Oct 31 03:38:40.698790 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.698756    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7386e379-d04b-44b9-8107-23139ac0f6c6-cert\") pod \"llmisvc-controller-manager-6466c554d8-jkdzt\" (UID: \"7386e379-d04b-44b9-8107-23139ac0f6c6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt"
Oct 31 03:38:40.698924 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.698792    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbfqt\" (UniqueName: \"kubernetes.io/projected/7386e379-d04b-44b9-8107-23139ac0f6c6-kube-api-access-dbfqt\") pod \"llmisvc-controller-manager-6466c554d8-jkdzt\" (UID: \"7386e379-d04b-44b9-8107-23139ac0f6c6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt"
Oct 31 03:38:40.701441 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.701417    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7386e379-d04b-44b9-8107-23139ac0f6c6-cert\") pod \"llmisvc-controller-manager-6466c554d8-jkdzt\" (UID: \"7386e379-d04b-44b9-8107-23139ac0f6c6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt"
Oct 31 03:38:40.762072 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.762046    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbfqt\" (UniqueName: \"kubernetes.io/projected/7386e379-d04b-44b9-8107-23139ac0f6c6-kube-api-access-dbfqt\") pod \"llmisvc-controller-manager-6466c554d8-jkdzt\" (UID: \"7386e379-d04b-44b9-8107-23139ac0f6c6\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt"
Oct 31 03:38:40.877365 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:40.877251    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt"
Oct 31 03:38:41.245662 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:38:41.245632    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7386e379_d04b_44b9_8107_23139ac0f6c6.slice/crio-ab6733478e058633a06b05f60abf14183907058119faf17bfcdf96c7aec9cdb6 WatchSource:0}: Error finding container ab6733478e058633a06b05f60abf14183907058119faf17bfcdf96c7aec9cdb6: Status 404 returned error can't find the container with id ab6733478e058633a06b05f60abf14183907058119faf17bfcdf96c7aec9cdb6
Oct 31 03:38:41.246848 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:41.246818    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt"]
Oct 31 03:38:41.911508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:41.911473    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt" event={"ID":"7386e379-d04b-44b9-8107-23139ac0f6c6","Type":"ContainerStarted","Data":"ab6733478e058633a06b05f60abf14183907058119faf17bfcdf96c7aec9cdb6"}
Oct 31 03:38:43.920568 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:43.920536    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt" event={"ID":"7386e379-d04b-44b9-8107-23139ac0f6c6","Type":"ContainerStarted","Data":"b33312c07c1909aaf1932343ae7911bd166252d36096a69e143eedbad7e4ab5e"}
Oct 31 03:38:43.921051 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:43.920646    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt"
Oct 31 03:38:44.039855 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:38:44.039806    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt" podStartSLOduration=1.58958999 podStartE2EDuration="4.039792587s" totalImagesPullingTime="2.450202597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:38:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:38:44.039356197 +0000 UTC m=+620.624896461" watchObservedRunningTime="2026-10-31 03:38:44.039792587 +0000 UTC m=+620.625332849"
Oct 31 03:39:05.311756 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:39:05.311712    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:39:14.927636 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:14.927604    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-jkdzt"
Oct 31 03:39:35.314202 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:39:35.314159    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:39:50.179749 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.179716    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"]
Oct 31 03:39:50.188082 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.188052    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.238834 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.238803    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"]
Oct 31 03:39:50.276751 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.276722    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fc591ce-8543-4590-9dd5-9002226418d2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.276921 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.276756    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fc591ce-8543-4590-9dd5-9002226418d2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.276921 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.276795    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fc591ce-8543-4590-9dd5-9002226418d2-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.276921 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.276902    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fc591ce-8543-4590-9dd5-9002226418d2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.277049 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.276933    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fc591ce-8543-4590-9dd5-9002226418d2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.277049 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.276964    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtw2h\" (UniqueName: \"kubernetes.io/projected/6fc591ce-8543-4590-9dd5-9002226418d2-kube-api-access-wtw2h\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.288708 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.288685    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 31 03:39:50.288810 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.288790    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 31 03:39:50.293258 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.293244    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Oct 31 03:39:50.293258 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.293251    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5hbl9\""
Oct 31 03:39:50.378292 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.378241    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fc591ce-8543-4590-9dd5-9002226418d2-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.378292 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.378298    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fc591ce-8543-4590-9dd5-9002226418d2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.378531 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.378320    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fc591ce-8543-4590-9dd5-9002226418d2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.378531 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.378348    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtw2h\" (UniqueName: \"kubernetes.io/projected/6fc591ce-8543-4590-9dd5-9002226418d2-kube-api-access-wtw2h\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.378531 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.378414    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fc591ce-8543-4590-9dd5-9002226418d2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.378531 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.378437    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fc591ce-8543-4590-9dd5-9002226418d2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.378754 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.378732    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fc591ce-8543-4590-9dd5-9002226418d2-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.378815 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.378773    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fc591ce-8543-4590-9dd5-9002226418d2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.378857 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.378810    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fc591ce-8543-4590-9dd5-9002226418d2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.380787 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.380763    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fc591ce-8543-4590-9dd5-9002226418d2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.381067 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.381053    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fc591ce-8543-4590-9dd5-9002226418d2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.441543 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.441471    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtw2h\" (UniqueName: \"kubernetes.io/projected/6fc591ce-8543-4590-9dd5-9002226418d2-kube-api-access-wtw2h\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4\" (UID: \"6fc591ce-8543-4590-9dd5-9002226418d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.520716 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.520679    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:39:50.886424 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:39:50.886394    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6fc591ce_8543_4590_9dd5_9002226418d2.slice/crio-3a510d2109f457ff6bf220874b864f7bdf841fb7055ef326af2b929201b7cf73 WatchSource:0}: Error finding container 3a510d2109f457ff6bf220874b864f7bdf841fb7055ef326af2b929201b7cf73: Status 404 returned error can't find the container with id 3a510d2109f457ff6bf220874b864f7bdf841fb7055ef326af2b929201b7cf73
Oct 31 03:39:50.887880 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.887862    4926 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 03:39:50.888122 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:50.888100    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"]
Oct 31 03:39:51.185726 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:51.185644    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4" event={"ID":"6fc591ce-8543-4590-9dd5-9002226418d2","Type":"ContainerStarted","Data":"3a510d2109f457ff6bf220874b864f7bdf841fb7055ef326af2b929201b7cf73"}
Oct 31 03:39:53.195858 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:53.195822    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4" event={"ID":"6fc591ce-8543-4590-9dd5-9002226418d2","Type":"ContainerStarted","Data":"14ecc00e3f31d4b22a186ccd5d3503464c1602697f59ab0a346ccfa9f9f5b5c0"}
Oct 31 03:39:53.317256 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:53.317206    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4" podStartSLOduration=1.80108092 podStartE2EDuration="3.317192379s" totalImagesPullingTime="1.516111459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:39:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:39:53.316626334 +0000 UTC m=+689.902166596" watchObservedRunningTime="2026-10-31 03:39:53.317192379 +0000 UTC m=+689.902732640"
Oct 31 03:39:53.521286 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:39:53.521237    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:40:00.520878 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:00.520841    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:40:00.533315 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:00.533286    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:40:01.242465 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:01.242435    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:40:05.312807 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:40:05.312762    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:40:17.253715 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.253680    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"]
Oct 31 03:40:17.260217 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.260199    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.314038 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.314009    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wk9x4\""
Oct 31 03:40:17.314038 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.314016    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 31 03:40:17.418411 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.418378    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.418575 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.418424    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.418575 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.418506    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.418575 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.418551    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.418759 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.418574    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.418759 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.418688    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.418759 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.418719    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.418759 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.418749    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dab396a-6307-46dc-9251-52e4dbc7f289-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.418928 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.418822    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgdp5\" (UniqueName: \"kubernetes.io/projected/4dab396a-6307-46dc-9251-52e4dbc7f289-kube-api-access-jgdp5\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.457714 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.457685    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"]
Oct 31 03:40:17.519570 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.519488    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.519570 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.519533    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.519793 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.519611    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.519793 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.519647    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.519793 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.519672    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dab396a-6307-46dc-9251-52e4dbc7f289-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.519793 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.519708    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgdp5\" (UniqueName: \"kubernetes.io/projected/4dab396a-6307-46dc-9251-52e4dbc7f289-kube-api-access-jgdp5\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.519793 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.519778    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.520057 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.519815    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.520057 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.519846    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.520057 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.519954    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.520057 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.520018    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.520211 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.520187    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.520355 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.520337    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dab396a-6307-46dc-9251-52e4dbc7f289-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.520403 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.520343    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.522229 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.522210    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.522490 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.522470    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.582342 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.582310    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.582647 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.582614    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgdp5\" (UniqueName: \"kubernetes.io/projected/4dab396a-6307-46dc-9251-52e4dbc7f289-kube-api-access-jgdp5\") pod \"maas-default-gateway-openshift-default-7854949c9-c6k9r\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:17.873439 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:17.873351    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:18.292685 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:40:18.292651    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dab396a_6307_46dc_9251_52e4dbc7f289.slice/crio-947b386e255cfa99cbce7ef345d25926b6a8715e1d6c541b71bf421d0a700332 WatchSource:0}: Error finding container 947b386e255cfa99cbce7ef345d25926b6a8715e1d6c541b71bf421d0a700332: Status 404 returned error can't find the container with id 947b386e255cfa99cbce7ef345d25926b6a8715e1d6c541b71bf421d0a700332
Oct 31 03:40:18.294539 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:18.294516    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"]
Oct 31 03:40:18.302811 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:18.302785    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r" event={"ID":"4dab396a-6307-46dc-9251-52e4dbc7f289","Type":"ContainerStarted","Data":"947b386e255cfa99cbce7ef345d25926b6a8715e1d6c541b71bf421d0a700332"}
Oct 31 03:40:23.226755 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:23.226725    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:40:23.227051 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:23.226792    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:40:23.227051 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:23.226851    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:40:23.325448 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:23.325416    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r" event={"ID":"4dab396a-6307-46dc-9251-52e4dbc7f289","Type":"ContainerStarted","Data":"ba7d26a0fe5df3e1ae8ed47872ab8d5a57361185db8be00dc87b7085388710ce"}
Oct 31 03:40:23.463986 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:23.463927    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r" podStartSLOduration=1.5328587470000001 podStartE2EDuration="6.463907804s" totalImagesPullingTime="4.931049057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:40:23.462547886 +0000 UTC m=+720.048088148" watchObservedRunningTime="2026-10-31 03:40:23.463907804 +0000 UTC m=+720.049448072"
Oct 31 03:40:23.873693 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:23.873645    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:24.875511 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:24.875475    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-c6k9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Oct 31 03:40:24.875939 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:24.875528    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r" podUID="4dab396a-6307-46dc-9251-52e4dbc7f289" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Oct 31 03:40:25.874614 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:25.874580    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-c6k9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Oct 31 03:40:25.874809 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:25.874642    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r" podUID="4dab396a-6307-46dc-9251-52e4dbc7f289" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Oct 31 03:40:26.874055 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:26.874018    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-c6k9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Oct 31 03:40:26.874591 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:26.874078    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r" podUID="4dab396a-6307-46dc-9251-52e4dbc7f289" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Oct 31 03:40:27.873753 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:27.873721    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-c6k9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Oct 31 03:40:27.873937 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:27.873773    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r" podUID="4dab396a-6307-46dc-9251-52e4dbc7f289" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Oct 31 03:40:28.874497 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:28.874463    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-c6k9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Oct 31 03:40:28.874887 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:28.874517    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r" podUID="4dab396a-6307-46dc-9251-52e4dbc7f289" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Oct 31 03:40:29.873771 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:29.873735    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-c6k9r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused" start-of-body=
Oct 31 03:40:29.873943 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:29.873796    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r" podUID="4dab396a-6307-46dc-9251-52e4dbc7f289" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.33:15021/healthz/ready\": dial tcp 10.132.0.33:15021: connect: connection refused"
Oct 31 03:40:31.458464 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:31.458432    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:31.458973 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:31.458845    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:31.459799 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:31.459774    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:40:32.356774 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.356737    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"]
Oct 31 03:40:32.364626 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.364592    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.436248 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.436205    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.436248 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.436250    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.436500 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.436298    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djwh4\" (UniqueName: \"kubernetes.io/projected/637fff8e-91cf-44c3-b626-def342c072c9-kube-api-access-djwh4\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.436500 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.436428    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.436500 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.436471    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/637fff8e-91cf-44c3-b626-def342c072c9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.436660 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.436559    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.436660 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.436582    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.436660 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.436602    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/637fff8e-91cf-44c3-b626-def342c072c9-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.436811 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.436741    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/637fff8e-91cf-44c3-b626-def342c072c9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.472259 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.472226    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"]
Oct 31 03:40:32.537575 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.537545    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/637fff8e-91cf-44c3-b626-def342c072c9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.537808 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.537593    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.537808 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.537615    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.537808 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.537642    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djwh4\" (UniqueName: \"kubernetes.io/projected/637fff8e-91cf-44c3-b626-def342c072c9-kube-api-access-djwh4\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.537808 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.537687    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.537808 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.537719    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/637fff8e-91cf-44c3-b626-def342c072c9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.537808 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.537746    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.537808 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.537770    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.537808 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.537789    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/637fff8e-91cf-44c3-b626-def342c072c9-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.538249 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.538225    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.538332 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.538232    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.538475 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.538451    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/637fff8e-91cf-44c3-b626-def342c072c9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.538546 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.538478    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.538546 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.538529    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.540255 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.540236    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.540708 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.540691    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/637fff8e-91cf-44c3-b626-def342c072c9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.600460 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.600417    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/637fff8e-91cf-44c3-b626-def342c072c9-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.600607 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.600502    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djwh4\" (UniqueName: \"kubernetes.io/projected/637fff8e-91cf-44c3-b626-def342c072c9-kube-api-access-djwh4\") pod \"maas-default-gateway-openshift-default-7854949c9-md4xl\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:32.676974 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:32.676899    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:33.092747 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:40:33.092715    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod637fff8e_91cf_44c3_b626_def342c072c9.slice/crio-1f0c46fd7f51f38312a8b8186630245b4ce4f7739d1bbde05b1adc8a52e9dbf9 WatchSource:0}: Error finding container 1f0c46fd7f51f38312a8b8186630245b4ce4f7739d1bbde05b1adc8a52e9dbf9: Status 404 returned error can't find the container with id 1f0c46fd7f51f38312a8b8186630245b4ce4f7739d1bbde05b1adc8a52e9dbf9
Oct 31 03:40:33.094838 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:33.094817    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"]
Oct 31 03:40:33.094966 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:33.094935    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:40:33.095022 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:33.094995    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:40:33.095060 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:33.095029    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:40:33.376968 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:33.376875    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl" event={"ID":"637fff8e-91cf-44c3-b626-def342c072c9","Type":"ContainerStarted","Data":"026c1ca7f4e86df69857ca84bc4f1f114c5980b91e7296f978511f982cb5f39c"}
Oct 31 03:40:33.376968 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:33.376942    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl" event={"ID":"637fff8e-91cf-44c3-b626-def342c072c9","Type":"ContainerStarted","Data":"1f0c46fd7f51f38312a8b8186630245b4ce4f7739d1bbde05b1adc8a52e9dbf9"}
Oct 31 03:40:33.504841 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:33.504788    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl" podStartSLOduration=1.504774162 podStartE2EDuration="1.504774162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:40:33.503679144 +0000 UTC m=+730.089219406" watchObservedRunningTime="2026-10-31 03:40:33.504774162 +0000 UTC m=+730.090314423"
Oct 31 03:40:33.677612 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:33.677524    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:34.679213 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:34.679172    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-md4xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Oct 31 03:40:34.679638 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:34.679237    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl" podUID="637fff8e-91cf-44c3-b626-def342c072c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Oct 31 03:40:35.336609 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:40:35.336573    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:40:35.678061 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:35.677983    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-md4xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Oct 31 03:40:35.678061 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:35.678036    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl" podUID="637fff8e-91cf-44c3-b626-def342c072c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Oct 31 03:40:36.677452 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:36.677416    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-md4xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Oct 31 03:40:36.677818 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:36.677469    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl" podUID="637fff8e-91cf-44c3-b626-def342c072c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Oct 31 03:40:37.677792 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:37.677753    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-md4xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Oct 31 03:40:37.678300 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:37.677819    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl" podUID="637fff8e-91cf-44c3-b626-def342c072c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Oct 31 03:40:38.677561 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:38.677524    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-md4xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Oct 31 03:40:38.677736 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:38.677578    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl" podUID="637fff8e-91cf-44c3-b626-def342c072c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Oct 31 03:40:39.677447 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:39.677402    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-md4xl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused" start-of-body=
Oct 31 03:40:39.677847 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:39.677473    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl" podUID="637fff8e-91cf-44c3-b626-def342c072c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.34:15021/healthz/ready\": dial tcp 10.132.0.34:15021: connect: connection refused"
Oct 31 03:40:41.322702 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:41.322670    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:41.323201 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:41.323011    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:40:41.323922 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:40:41.323901    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:41:05.311171 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:41:05.311090    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:41:35.311731 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:41:35.311691    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:42:05.313499 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:42:05.313460    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:42:35.311436 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:42:35.311387    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:43:05.314110 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:43:05.313998    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:43:13.118376 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.118342    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"]
Oct 31 03:43:13.121886 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.121870    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.176640 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.176587    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-d7b7bc-openshift-default-dockercfg-nszpp\""
Oct 31 03:43:13.228880 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.228846    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"]
Oct 31 03:43:13.295665 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.295628    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-data\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.295830 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.295680    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-podinfo\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.295830 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.295745    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-workload-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.295830 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.295779    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istiod-ca-cert\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.295830 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.295794    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj7h8\" (UniqueName: \"kubernetes.io/projected/7ce8937f-8f05-4e63-85bb-5ee282b5367b-kube-api-access-mj7h8\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.295830 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.295813    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-workload-certs\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.295993 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.295877    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-token\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.296026 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.295994    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-credential-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.296026 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.296021    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-envoy\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.397142 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.397054    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-podinfo\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.397142 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.397093    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-workload-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.397142 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.397123    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istiod-ca-cert\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.397142 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.397138    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj7h8\" (UniqueName: \"kubernetes.io/projected/7ce8937f-8f05-4e63-85bb-5ee282b5367b-kube-api-access-mj7h8\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.397506 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.397157    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-workload-certs\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.397506 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.397185    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-token\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.397506 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.397230    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-credential-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.397506 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.397253    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-envoy\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.397506 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.397368    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-data\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.397790 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.397759    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-workload-certs\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.397857 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.397817    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-workload-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.397936 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.397911    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-credential-socket\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.398002 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.397938    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-data\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.398529 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.398450    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istiod-ca-cert\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.399896 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.399875    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-podinfo\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.399981 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.399957    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-envoy\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.460493 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.460465    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj7h8\" (UniqueName: \"kubernetes.io/projected/7ce8937f-8f05-4e63-85bb-5ee282b5367b-kube-api-access-mj7h8\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.460493 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.460464    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-token\") pod \"e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") " pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:13.734800 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:13.734770    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:14.163203 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:43:14.163169    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce8937f_8f05_4e63_85bb_5ee282b5367b.slice/crio-c2b47f56b149f7c2c9f38e4bfc3f556f75af6611a4d191b7fd7219c4f69dd146 WatchSource:0}: Error finding container c2b47f56b149f7c2c9f38e4bfc3f556f75af6611a4d191b7fd7219c4f69dd146: Status 404 returned error can't find the container with id c2b47f56b149f7c2c9f38e4bfc3f556f75af6611a4d191b7fd7219c4f69dd146
Oct 31 03:43:14.164938 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.164910    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"]
Oct 31 03:43:14.165458 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.165427    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:43:14.165520 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.165500    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:43:14.165554 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.165533    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:43:14.424558 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.424460    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"]
Oct 31 03:43:14.427852 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.427835    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:14.482494 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.482039    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-412ba5-serving-cert\""
Oct 31 03:43:14.486674 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.486647    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"]
Oct 31 03:43:14.606701 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.606666    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:14.606855 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.606725    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzw4x\" (UniqueName: \"kubernetes.io/projected/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-kube-api-access-fzw4x\") pod \"maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:14.606855 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.606825    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-metrics-certs\") pod \"maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:14.606963 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.606861    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-maas-api-tls\") pod \"maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:14.708383 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.708349    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzw4x\" (UniqueName: \"kubernetes.io/projected/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-kube-api-access-fzw4x\") pod \"maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:14.708556 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.708390    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-metrics-certs\") pod \"maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:14.708556 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.708416    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-maas-api-tls\") pod \"maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:14.708556 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.708474    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:14.709184 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.709161    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:14.711023 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.711000    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-maas-api-tls\") pod \"maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:14.711106 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.711031    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-metrics-certs\") pod \"maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:14.770989 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:14.770958    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzw4x\" (UniqueName: \"kubernetes.io/projected/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-kube-api-access-fzw4x\") pod \"maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:15.043497 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:15.043415    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:15.059375 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:15.059348    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8" event={"ID":"7ce8937f-8f05-4e63-85bb-5ee282b5367b","Type":"ContainerStarted","Data":"8681d42d6bb99b5db00bef4f8a5917b3f14aa67df0ec6ea0c9e93ac0ed4fc6bf"}
Oct 31 03:43:15.059476 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:15.059402    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8" event={"ID":"7ce8937f-8f05-4e63-85bb-5ee282b5367b","Type":"ContainerStarted","Data":"c2b47f56b149f7c2c9f38e4bfc3f556f75af6611a4d191b7fd7219c4f69dd146"}
Oct 31 03:43:15.184284 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:15.184212    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8" podStartSLOduration=2.184197255 podStartE2EDuration="2.184197255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:15.18363732 +0000 UTC m=+891.769177583" watchObservedRunningTime="2026-10-31 03:43:15.184197255 +0000 UTC m=+891.769737514"
Oct 31 03:43:15.409941 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:43:15.409897    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73c02cf_bb22_4571_b20b_5445ac0f9c8b.slice/crio-e41c520f86e122365a7a2538b34c7cbc861e3557571586e5907778c72367e81b WatchSource:0}: Error finding container e41c520f86e122365a7a2538b34c7cbc861e3557571586e5907778c72367e81b: Status 404 returned error can't find the container with id e41c520f86e122365a7a2538b34c7cbc861e3557571586e5907778c72367e81b
Oct 31 03:43:15.411659 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:15.411634    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"]
Oct 31 03:43:15.735845 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:15.735813    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:15.741337 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:15.741312    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:16.074796 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:16.073109    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx" event={"ID":"b73c02cf-bb22-4571-b20b-5445ac0f9c8b","Type":"ContainerStarted","Data":"e41c520f86e122365a7a2538b34c7cbc861e3557571586e5907778c72367e81b"}
Oct 31 03:43:16.074796 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:16.073179    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:16.074796 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:16.074720    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:43:17.077305 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:17.077252    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx" event={"ID":"b73c02cf-bb22-4571-b20b-5445ac0f9c8b","Type":"ContainerStarted","Data":"725919db86e5b5fdf957dbc964a9928bc78c58d6739b03027bdf473e6c376928"}
Oct 31 03:43:17.077690 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:17.077457    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:17.198321 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:17.198255    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx" podStartSLOduration=2.281402269 podStartE2EDuration="3.198240937s" totalImagesPullingTime="916.838668ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:17.196981986 +0000 UTC m=+893.782522250" watchObservedRunningTime="2026-10-31 03:43:17.198240937 +0000 UTC m=+893.783781200"
Oct 31 03:43:19.296200 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:19.296167    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"]
Oct 31 03:43:19.299439 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:19.299424    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:19.367340 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:19.367304    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-68aa3e-serving-cert\""
Oct 31 03:43:19.410914 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:19.410883    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"]
Oct 31 03:43:19.452524 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:19.452495    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-metrics-certs\") pod \"maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:19.452690 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:19.452640    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-maas-api-tls\") pod \"maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:19.452690 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:19.452672    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q5qn\" (UniqueName: \"kubernetes.io/projected/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-kube-api-access-4q5qn\") pod \"maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:19.452761 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:19.452703    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:19.553992 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:19.553908    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-maas-api-tls\") pod \"maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:19.553992 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:19.553945    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q5qn\" (UniqueName: \"kubernetes.io/projected/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-kube-api-access-4q5qn\") pod \"maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:19.553992 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:19.553970    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:19.553992 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:19.553992    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-metrics-certs\") pod \"maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:19.554361 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:43:19.554095    4926 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-serving-cert: secret "maas-api-e2e-worker-w4-68aa3e-serving-cert" not found
Oct 31 03:43:19.554361 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:43:19.554189    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-maas-api-tls podName:2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed nodeName:}" failed. No retries permitted until 2026-10-31 03:43:20.054165912 +0000 UTC m=+896.639706154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-maas-api-tls") pod "maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn" (UID: "2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed") : secret "maas-api-e2e-worker-w4-68aa3e-serving-cert" not found
Oct 31 03:43:19.555022 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:19.554995    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:19.556648 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:19.556629    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-metrics-certs\") pod \"maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:19.618923 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:19.618886    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q5qn\" (UniqueName: \"kubernetes.io/projected/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-kube-api-access-4q5qn\") pod \"maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:20.058939 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:20.058904    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-maas-api-tls\") pod \"maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:20.061428 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:20.061408    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-maas-api-tls\") pod \"maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:20.212631 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:20.212586    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:20.595679 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:43:20.595643    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c5bc95d_4ff6_4049_8e4c_ec0725ca68ed.slice/crio-bba784583d2d232c9805faaf4175346d2eb4e0ef48e6b083605620d44a17d61b WatchSource:0}: Error finding container bba784583d2d232c9805faaf4175346d2eb4e0ef48e6b083605620d44a17d61b: Status 404 returned error can't find the container with id bba784583d2d232c9805faaf4175346d2eb4e0ef48e6b083605620d44a17d61b
Oct 31 03:43:20.597203 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:20.597178    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"]
Oct 31 03:43:21.100965 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:21.100933    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn" event={"ID":"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed","Type":"ContainerStarted","Data":"bba784583d2d232c9805faaf4175346d2eb4e0ef48e6b083605620d44a17d61b"}
Oct 31 03:43:22.106557 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:22.106520    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn" event={"ID":"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed","Type":"ContainerStarted","Data":"4d6c70c0c662671945f17d5e1ed82fea5a558d0d3d30480164ee0dbc0e8189fb"}
Oct 31 03:43:22.106960 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:22.106586    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:22.229032 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:22.228987    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn" podStartSLOduration=2.390962305 podStartE2EDuration="3.22897287s" totalImagesPullingTime="838.010565ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:22.227840967 +0000 UTC m=+898.813381229" watchObservedRunningTime="2026-10-31 03:43:22.22897287 +0000 UTC m=+898.814513129"
Oct 31 03:43:23.088155 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:23.088127    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 03:43:23.600531 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:23.600499    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 03:43:23.601069 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:23.601049    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 03:43:24.393178 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:24.393139    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"]
Oct 31 03:43:24.397041 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:24.397020    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:24.398885 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:24.398855    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-maas-api-tls\") pod \"maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:24.398976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:24.398908    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpsv8\" (UniqueName: \"kubernetes.io/projected/74e17325-bf76-4639-9475-b54f780ed429-kube-api-access-kpsv8\") pod \"maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:24.399061 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:24.399040    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-metrics-certs\") pod \"maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:24.399161 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:24.399142    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74e17325-bf76-4639-9475-b54f780ed429-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:24.452595 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:24.452556    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"]
Oct 31 03:43:24.499986 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:24.499950    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-maas-api-tls\") pod \"maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:24.499986 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:24.499990    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpsv8\" (UniqueName: \"kubernetes.io/projected/74e17325-bf76-4639-9475-b54f780ed429-kube-api-access-kpsv8\") pod \"maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:24.500201 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:24.500016    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-metrics-certs\") pod \"maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:24.500201 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:24.500052    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74e17325-bf76-4639-9475-b54f780ed429-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:24.500770 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:24.500751    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74e17325-bf76-4639-9475-b54f780ed429-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:24.501602 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:24.501584    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-ae8e27-serving-cert\""
Oct 31 03:43:24.502769 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:24.502747    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-metrics-certs\") pod \"maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:24.510487 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:43:24.510466    4926 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-serving-cert: secret "maas-api-e2e-worker-w0-ae8e27-serving-cert" not found
Oct 31 03:43:24.510582 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:43:24.510521    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-maas-api-tls podName:74e17325-bf76-4639-9475-b54f780ed429 nodeName:}" failed. No retries permitted until 2026-10-31 03:43:25.010505881 +0000 UTC m=+901.596046120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-maas-api-tls") pod "maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4" (UID: "74e17325-bf76-4639-9475-b54f780ed429") : secret "maas-api-e2e-worker-w0-ae8e27-serving-cert" not found
Oct 31 03:43:24.563246 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:24.563210    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpsv8\" (UniqueName: \"kubernetes.io/projected/74e17325-bf76-4639-9475-b54f780ed429-kube-api-access-kpsv8\") pod \"maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:25.107885 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:25.107844    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-maas-api-tls\") pod \"maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:25.110416 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:25.110398    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-maas-api-tls\") pod \"maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:25.309279 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:25.309244    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:25.675181 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:43:25.675142    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e17325_bf76_4639_9475_b54f780ed429.slice/crio-7982cb81b9ac074162ace0f226fa2497632189db3f519da046746012d57f734e WatchSource:0}: Error finding container 7982cb81b9ac074162ace0f226fa2497632189db3f519da046746012d57f734e: Status 404 returned error can't find the container with id 7982cb81b9ac074162ace0f226fa2497632189db3f519da046746012d57f734e
Oct 31 03:43:25.677299 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:25.677148    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"]
Oct 31 03:43:26.122814 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:26.122778    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4" event={"ID":"74e17325-bf76-4639-9475-b54f780ed429","Type":"ContainerStarted","Data":"7982cb81b9ac074162ace0f226fa2497632189db3f519da046746012d57f734e"}
Oct 31 03:43:27.128440 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:27.128402    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4" event={"ID":"74e17325-bf76-4639-9475-b54f780ed429","Type":"ContainerStarted","Data":"7c8dc3220d4c979dfd23aa746297408d4c36935963d4c0aa6c3cfa7a9d3b73a7"}
Oct 31 03:43:27.128816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:27.128478    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:27.266435 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:27.266387    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4" podStartSLOduration=2.402553797 podStartE2EDuration="3.266373083s" totalImagesPullingTime="863.819286ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:27.262310162 +0000 UTC m=+903.847850421" watchObservedRunningTime="2026-10-31 03:43:27.266373083 +0000 UTC m=+903.851913345"
Oct 31 03:43:28.115471 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:28.115445    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 03:43:33.137779 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:33.137748    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 03:43:35.315815 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:43:35.315780    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:43:45.183651 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.183620    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"]
Oct 31 03:43:45.187188 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.187166    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:45.245094 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.244139    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-d7b7bc-serving-cert\""
Oct 31 03:43:45.245094 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.244999    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"]
Oct 31 03:43:45.282713 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.282681    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-metrics-certs\") pod \"maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:45.282868 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.282724    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfcnq\" (UniqueName: \"kubernetes.io/projected/0664091e-7353-433c-bd14-76c8b1b6d795-kube-api-access-kfcnq\") pod \"maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:45.282868 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.282823    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-maas-api-tls\") pod \"maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:45.282969 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.282950    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0664091e-7353-433c-bd14-76c8b1b6d795-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:45.384287 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.384234    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-metrics-certs\") pod \"maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:45.384473 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.384312    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfcnq\" (UniqueName: \"kubernetes.io/projected/0664091e-7353-433c-bd14-76c8b1b6d795-kube-api-access-kfcnq\") pod \"maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:45.384473 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.384371    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-maas-api-tls\") pod \"maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:45.384473 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.384442    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0664091e-7353-433c-bd14-76c8b1b6d795-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:45.384651 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:43:45.384523    4926 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-serving-cert: secret "maas-api-e2e-worker-w1-d7b7bc-serving-cert" not found
Oct 31 03:43:45.384651 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:43:45.384602    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-maas-api-tls podName:0664091e-7353-433c-bd14-76c8b1b6d795 nodeName:}" failed. No retries permitted until 2026-10-31 03:43:45.884580624 +0000 UTC m=+922.470120864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-maas-api-tls") pod "maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk" (UID: "0664091e-7353-433c-bd14-76c8b1b6d795") : secret "maas-api-e2e-worker-w1-d7b7bc-serving-cert" not found
Oct 31 03:43:45.385096 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.385080    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0664091e-7353-433c-bd14-76c8b1b6d795-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:45.386804 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.386788    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-metrics-certs\") pod \"maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:45.461910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.461885    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfcnq\" (UniqueName: \"kubernetes.io/projected/0664091e-7353-433c-bd14-76c8b1b6d795-kube-api-access-kfcnq\") pod \"maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:45.889894 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.889809    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-maas-api-tls\") pod \"maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:45.892429 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:45.892409    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-maas-api-tls\") pod \"maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:46.098895 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:46.098859    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:46.467721 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:43:46.467684    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0664091e_7353_433c_bd14_76c8b1b6d795.slice/crio-8c5d1f3ea28c477e25bcbbc96d1b8a1a493aac356a15ae4cf2c5a685280705ea WatchSource:0}: Error finding container 8c5d1f3ea28c477e25bcbbc96d1b8a1a493aac356a15ae4cf2c5a685280705ea: Status 404 returned error can't find the container with id 8c5d1f3ea28c477e25bcbbc96d1b8a1a493aac356a15ae4cf2c5a685280705ea
Oct 31 03:43:46.470503 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:46.470465    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"]
Oct 31 03:43:47.214989 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:47.214954    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk" event={"ID":"0664091e-7353-433c-bd14-76c8b1b6d795","Type":"ContainerStarted","Data":"8c5d1f3ea28c477e25bcbbc96d1b8a1a493aac356a15ae4cf2c5a685280705ea"}
Oct 31 03:43:48.221059 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:48.221020    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk" event={"ID":"0664091e-7353-433c-bd14-76c8b1b6d795","Type":"ContainerStarted","Data":"e98232df5eb9824bcbac8a2290064ce47dfb9687da454f130920f731fad453b5"}
Oct 31 03:43:48.221547 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:48.221155    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:48.343841 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:48.343791    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk" podStartSLOduration=2.297493375 podStartE2EDuration="3.343778254s" totalImagesPullingTime="1.046284879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:48.342032171 +0000 UTC m=+924.927572433" watchObservedRunningTime="2026-10-31 03:43:48.343778254 +0000 UTC m=+924.929318516"
Oct 31 03:43:52.637099 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.637061    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"]
Oct 31 03:43:52.640762 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.640743    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.694726 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.694696    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"]
Oct 31 03:43:52.744570 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.744538    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-412ba5\"/\"openshift-service-ca.crt\""
Oct 31 03:43:52.744750 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.744577    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-412ba5\"/\"kube-root-ca.crt\""
Oct 31 03:43:52.744750 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.744576    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-412ba5\"/\"default-dockercfg-rq9bv\""
Oct 31 03:43:52.746244 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.746228    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-412ba5\"/\"simulator-w6-412ba5-kserve-self-signed-certs\""
Oct 31 03:43:52.752641 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.752618    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-home\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.752712 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.752652    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cfgh\" (UniqueName: \"kubernetes.io/projected/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-kube-api-access-8cfgh\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.752712 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.752693    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-tmp-dir\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.752776 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.752737    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-tls-certs\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.752841 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.752827    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-model-cache\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.752876 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.752868    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-dshm\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.853413 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.853383    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-dshm\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.853560 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.853429    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-home\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.853560 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.853447    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cfgh\" (UniqueName: \"kubernetes.io/projected/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-kube-api-access-8cfgh\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.853560 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.853487    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-tmp-dir\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.853560 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.853504    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-tls-certs\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.853788 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.853562    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-model-cache\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.853967 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.853941    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-tmp-dir\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.854035 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.853973    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-home\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.854035 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.853988    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-model-cache\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.855761 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.855739    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-dshm\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.856166 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.856147    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-tls-certs\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.916432 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.916362    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cfgh\" (UniqueName: \"kubernetes.io/projected/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-kube-api-access-8cfgh\") pod \"simulator-w6-412ba5-kserve-849678b49d-jf46j\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") " pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:52.952066 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:52.952027    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:43:53.327072 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:43:53.327043    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf97017f6_52c4_4d27_9aa7_5045b9ac85ee.slice/crio-a3a1bd43c961326de78f60c8f1b40371474aeecbb8a3319c8fff295beb22d054 WatchSource:0}: Error finding container a3a1bd43c961326de78f60c8f1b40371474aeecbb8a3319c8fff295beb22d054: Status 404 returned error can't find the container with id a3a1bd43c961326de78f60c8f1b40371474aeecbb8a3319c8fff295beb22d054
Oct 31 03:43:53.330527 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:53.330496    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"]
Oct 31 03:43:54.230132 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:54.230103    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 03:43:54.251065 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:54.251035    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j" event={"ID":"f97017f6-52c4-4d27-9aa7-5045b9ac85ee","Type":"ContainerStarted","Data":"18bbb12ec7ffebd2133be31b7aaaf918e3d4221354019bf4a216768570e98ba4"}
Oct 31 03:43:54.251225 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:54.251076    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j" event={"ID":"f97017f6-52c4-4d27-9aa7-5045b9ac85ee","Type":"ContainerStarted","Data":"a3a1bd43c961326de78f60c8f1b40371474aeecbb8a3319c8fff295beb22d054"}
Oct 31 03:43:54.472154 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:54.472064    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j" podStartSLOduration=2.472045733 podStartE2EDuration="2.472045733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:43:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:43:54.471008178 +0000 UTC m=+931.056548439" watchObservedRunningTime="2026-10-31 03:43:54.472045733 +0000 UTC m=+931.057585999"
Oct 31 03:43:54.953175 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:43:54.953144    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:44:02.953144 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:02.953064    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:44:02.965914 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:02.965883    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:44:03.304864 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:03.304777    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:44:04.298838 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:04.298811    4926 generic.go:356] "Generic (PLEG): container finished" podID="7ce8937f-8f05-4e63-85bb-5ee282b5367b" containerID="8681d42d6bb99b5db00bef4f8a5917b3f14aa67df0ec6ea0c9e93ac0ed4fc6bf" exitCode=0
Oct 31 03:44:04.299360 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:04.298876    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8" event={"ID":"7ce8937f-8f05-4e63-85bb-5ee282b5367b","Type":"ContainerDied","Data":"8681d42d6bb99b5db00bef4f8a5917b3f14aa67df0ec6ea0c9e93ac0ed4fc6bf"}
Oct 31 03:44:04.299461 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:04.299438    4926 scope.go:122] "RemoveContainer" containerID="8681d42d6bb99b5db00bef4f8a5917b3f14aa67df0ec6ea0c9e93ac0ed4fc6bf"
Oct 31 03:44:04.302790 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:04.302765    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:44:04.302912 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:04.302832    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:44:04.302912 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:04.302870    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:44:05.306767 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:05.306717    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8" event={"ID":"7ce8937f-8f05-4e63-85bb-5ee282b5367b","Type":"ContainerStarted","Data":"c1ff09a2d939d10285ec82248550f64742fc22d533c86f343f37e118a58ae16c"}
Oct 31 03:44:05.339468 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:44:05.339436    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:44:05.735201 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:05.734866    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:44:05.735432 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:05.735215    4926 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Oct 31 03:44:05.735432 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:05.735290    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8" podUID="7ce8937f-8f05-4e63-85bb-5ee282b5367b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Oct 31 03:44:06.737298 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:06.736729    4926 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Oct 31 03:44:06.737298 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:06.736840    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8" podUID="7ce8937f-8f05-4e63-85bb-5ee282b5367b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Oct 31 03:44:07.735757 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:07.735715    4926 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Oct 31 03:44:07.735979 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:07.735786    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8" podUID="7ce8937f-8f05-4e63-85bb-5ee282b5367b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Oct 31 03:44:08.735905 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:08.735875    4926 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Oct 31 03:44:08.736378 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:08.736134    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8" podUID="7ce8937f-8f05-4e63-85bb-5ee282b5367b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Oct 31 03:44:09.735763 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:09.735723    4926 patch_prober.go:29] interesting pod/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body=
Oct 31 03:44:09.736039 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:09.736007    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8" podUID="7ce8937f-8f05-4e63-85bb-5ee282b5367b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused"
Oct 31 03:44:10.739052 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:10.739020    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:44:10.739556 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:10.739298    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:44:10.740110 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:10.740087    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 03:44:17.176120 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.176078    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"]
Oct 31 03:44:17.179999 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.179983    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.211477 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.211437    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp6lk\" (UniqueName: \"kubernetes.io/projected/a5ad5303-15e7-422a-a144-5598d90f33db-kube-api-access-rp6lk\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.211635 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.211556    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-home\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.211693 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.211639    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ad5303-15e7-422a-a144-5598d90f33db-tls-certs\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.211730 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.211709    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-tmp-dir\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.211767 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.211754    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-dshm\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.211800 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.211785    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-model-cache\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.234707 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.234674    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"]
Oct 31 03:44:17.284115 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.284089    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-68aa3e\"/\"openshift-service-ca.crt\""
Oct 31 03:44:17.284355 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.284334    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-68aa3e\"/\"kube-root-ca.crt\""
Oct 31 03:44:17.284435 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.284360    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-68aa3e\"/\"default-dockercfg-bjffd\""
Oct 31 03:44:17.285887 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.285861    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-68aa3e\"/\"simulator-w4-68aa3e-kserve-self-signed-certs\""
Oct 31 03:44:17.312607 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.312568    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ad5303-15e7-422a-a144-5598d90f33db-tls-certs\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.312607 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.312614    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-tmp-dir\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.312768 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.312644    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-dshm\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.312768 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.312672    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-model-cache\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.312768 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.312696    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp6lk\" (UniqueName: \"kubernetes.io/projected/a5ad5303-15e7-422a-a144-5598d90f33db-kube-api-access-rp6lk\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.312768 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.312748    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-home\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.313102 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.313083    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-model-cache\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.313182 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.313140    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-tmp-dir\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.313222 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.313181    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-home\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.314992 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.314968    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-dshm\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.315381 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.315363    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ad5303-15e7-422a-a144-5598d90f33db-tls-certs\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.376089 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.376055    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp6lk\" (UniqueName: \"kubernetes.io/projected/a5ad5303-15e7-422a-a144-5598d90f33db-kube-api-access-rp6lk\") pod \"simulator-w4-68aa3e-kserve-6fc565d487-cwzwj\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") " pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.491294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.491249    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:17.865109 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:44:17.865078    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5ad5303_15e7_422a_a144_5598d90f33db.slice/crio-3a6b706d2259aca05793e0694e58f4f275b24041e8f7a21b5315293a4b15318c WatchSource:0}: Error finding container 3a6b706d2259aca05793e0694e58f4f275b24041e8f7a21b5315293a4b15318c: Status 404 returned error can't find the container with id 3a6b706d2259aca05793e0694e58f4f275b24041e8f7a21b5315293a4b15318c
Oct 31 03:44:17.867755 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:17.867725    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"]
Oct 31 03:44:18.387848 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:18.387807    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj" event={"ID":"a5ad5303-15e7-422a-a144-5598d90f33db","Type":"ContainerStarted","Data":"12a1fc54829057d527705be7f2e3da7400144278df9c1dbc09d456b68f989c35"}
Oct 31 03:44:18.387848 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:18.387849    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj" event={"ID":"a5ad5303-15e7-422a-a144-5598d90f33db","Type":"ContainerStarted","Data":"3a6b706d2259aca05793e0694e58f4f275b24041e8f7a21b5315293a4b15318c"}
Oct 31 03:44:18.492107 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:18.492075    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:18.508509 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:18.508463    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj" podStartSLOduration=1.508451054 podStartE2EDuration="1.508451054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:44:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:44:18.507780096 +0000 UTC m=+955.093320357" watchObservedRunningTime="2026-10-31 03:44:18.508451054 +0000 UTC m=+955.093991315"
Oct 31 03:44:27.491432 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:27.491396    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:27.503687 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:27.503656    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:28.445105 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:28.445079    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:44:35.313698 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:44:35.313663    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:44:38.554041 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.554005    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"]
Oct 31 03:44:38.557518 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.557502    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.595731 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.595704    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-tmp-dir\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.595899 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.595755    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svcx6\" (UniqueName: \"kubernetes.io/projected/564888f8-35bb-4028-8040-110c959c9b19-kube-api-access-svcx6\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.595899 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.595784    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-model-cache\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.595899 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.595882    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-home\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.596055 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.595917    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/564888f8-35bb-4028-8040-110c959c9b19-tls-certs\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.596114 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.596077    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-dshm\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.612548 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.612523    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"]
Oct 31 03:44:38.662371 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.662344    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-537e34\"/\"openshift-service-ca.crt\""
Oct 31 03:44:38.662528 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.662444    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-537e34\"/\"simulator-w3-537e34-kserve-self-signed-certs\""
Oct 31 03:44:38.662528 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.662484    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-537e34\"/\"kube-root-ca.crt\""
Oct 31 03:44:38.662762 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.662748    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-537e34\"/\"default-dockercfg-9fnbs\""
Oct 31 03:44:38.696806 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.696780    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-model-cache\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.696986 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.696811    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-home\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.696986 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.696831    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/564888f8-35bb-4028-8040-110c959c9b19-tls-certs\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.696986 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.696882    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-dshm\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.696986 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.696954    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-tmp-dir\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.697193 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.697009    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svcx6\" (UniqueName: \"kubernetes.io/projected/564888f8-35bb-4028-8040-110c959c9b19-kube-api-access-svcx6\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.697292 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.697254    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-model-cache\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.697374 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.697345    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-tmp-dir\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.698252 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.698209    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-home\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.700240 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.700220    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/564888f8-35bb-4028-8040-110c959c9b19-tls-certs\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.702778 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.702750    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-dshm\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.780919 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.780874    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svcx6\" (UniqueName: \"kubernetes.io/projected/564888f8-35bb-4028-8040-110c959c9b19-kube-api-access-svcx6\") pod \"simulator-w3-537e34-kserve-9c8459cff-n2znw\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") " pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:38.868641 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:38.868558    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:39.256865 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:44:39.256833    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod564888f8_35bb_4028_8040_110c959c9b19.slice/crio-21554e32e73743d263944a51e3af54a7501ab39950a0db216bb9a4c952cf4424 WatchSource:0}: Error finding container 21554e32e73743d263944a51e3af54a7501ab39950a0db216bb9a4c952cf4424: Status 404 returned error can't find the container with id 21554e32e73743d263944a51e3af54a7501ab39950a0db216bb9a4c952cf4424
Oct 31 03:44:39.257641 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:39.257612    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"]
Oct 31 03:44:39.487552 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:39.487522    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" event={"ID":"564888f8-35bb-4028-8040-110c959c9b19","Type":"ContainerStarted","Data":"f0692352bbbc5d76ee0616daec4d58c37059fc518e73ebf590f6203ddef530e3"}
Oct 31 03:44:39.487724 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:39.487560    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" event={"ID":"564888f8-35bb-4028-8040-110c959c9b19","Type":"ContainerStarted","Data":"21554e32e73743d263944a51e3af54a7501ab39950a0db216bb9a4c952cf4424"}
Oct 31 03:44:39.607813 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:39.607706    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" podStartSLOduration=1.607693148 podStartE2EDuration="1.607693148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:44:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:44:39.606505053 +0000 UTC m=+976.192045315" watchObservedRunningTime="2026-10-31 03:44:39.607693148 +0000 UTC m=+976.193233462"
Oct 31 03:44:39.869026 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:39.868933    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:48.869083 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:48.869050    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:48.881348 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:48.881324    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:49.545692 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.545662    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:44:49.732984 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.732957    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"]
Oct 31 03:44:49.736667 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.736649    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.790769 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.790731    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-ae8e27\"/\"kube-root-ca.crt\""
Oct 31 03:44:49.790948 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.790924    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-ae8e27\"/\"default-dockercfg-kh5kd\""
Oct 31 03:44:49.791089 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.790969    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-ae8e27\"/\"simulator-w0-ae8e27-kserve-self-signed-certs\""
Oct 31 03:44:49.791089 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.790987    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-ae8e27\"/\"openshift-service-ca.crt\""
Oct 31 03:44:49.791490 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.791470    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"]
Oct 31 03:44:49.801514 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.801457    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-tls-certs\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.801514 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.801505    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-tmp-dir\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.801638 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.801573    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-model-cache\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.801638 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.801621    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-home\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.801715 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.801680    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bv2k\" (UniqueName: \"kubernetes.io/projected/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-kube-api-access-4bv2k\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.801756 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.801715    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-dshm\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.902298 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.902234    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-tls-certs\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.902795 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.902322    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-tmp-dir\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.902795 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.902351    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-model-cache\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.902795 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.902410    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-home\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.902795 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.902452    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bv2k\" (UniqueName: \"kubernetes.io/projected/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-kube-api-access-4bv2k\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.902795 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.902501    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-dshm\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.903041 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.902815    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-tmp-dir\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.903041 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.902828    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-model-cache\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.903041 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.902868    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-home\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.904817 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.904796    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-dshm\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.905071 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.905051    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-tls-certs\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:49.965379 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:49.965353    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bv2k\" (UniqueName: \"kubernetes.io/projected/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-kube-api-access-4bv2k\") pod \"simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") " pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:50.048278 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:50.048228    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:50.425005 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:44:50.424974    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda015eb6f_0f4c_4c57_b2aa_dc06e0fdf228.slice/crio-d5eca5e57079588fab30da2a46ea0d8411f548982f70396a8e316d3c7150b924 WatchSource:0}: Error finding container d5eca5e57079588fab30da2a46ea0d8411f548982f70396a8e316d3c7150b924: Status 404 returned error can't find the container with id d5eca5e57079588fab30da2a46ea0d8411f548982f70396a8e316d3c7150b924
Oct 31 03:44:50.427310 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:50.427249    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"]
Oct 31 03:44:50.541706 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:50.541665    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" event={"ID":"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228","Type":"ContainerStarted","Data":"1d30889e6ccc2fcac9dbbb123f2d4f6806aa14d3c043f951ae1c146a0d974185"}
Oct 31 03:44:50.541883 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:50.541716    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" event={"ID":"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228","Type":"ContainerStarted","Data":"d5eca5e57079588fab30da2a46ea0d8411f548982f70396a8e316d3c7150b924"}
Oct 31 03:44:50.661092 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:50.661042    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" podStartSLOduration=1.661029533 podStartE2EDuration="1.661029533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:44:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:44:50.660568244 +0000 UTC m=+987.246108506" watchObservedRunningTime="2026-10-31 03:44:50.661029533 +0000 UTC m=+987.246569796"
Oct 31 03:44:51.048336 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:51.048305    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:44:54.847400 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:54.847364    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"]
Oct 31 03:44:54.851035 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:54.851016    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:54.905631 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:54.905601    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"]
Oct 31 03:44:54.954998 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:54.954965    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-model-cache\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:54.955163 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:54.955010    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-home\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:54.955214 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:54.955163    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f198d8-39a2-4c72-87de-516e077d47bd-tls-certs\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:54.955214 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:54.955197    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-tmp-dir\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:54.955313 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:54.955231    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-dshm\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:54.955313 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:54.955252    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj7gn\" (UniqueName: \"kubernetes.io/projected/a5f198d8-39a2-4c72-87de-516e077d47bd-kube-api-access-tj7gn\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:54.971874 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:54.971850    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-412ba5\"/\"premium-w6-412ba5-kserve-self-signed-certs\""
Oct 31 03:44:55.056563 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:55.056526    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-model-cache\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:55.056754 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:55.056586    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-home\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:55.056754 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:55.056682    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f198d8-39a2-4c72-87de-516e077d47bd-tls-certs\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:55.056754 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:55.056721    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-tmp-dir\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:55.056927 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:55.056846    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-dshm\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:55.056927 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:55.056875    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj7gn\" (UniqueName: \"kubernetes.io/projected/a5f198d8-39a2-4c72-87de-516e077d47bd-kube-api-access-tj7gn\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:55.057035 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:55.057012    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-model-cache\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:55.057219 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:55.057189    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-tmp-dir\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:55.057219 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:55.057214    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-home\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:55.059421 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:55.059398    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-dshm\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:55.059716 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:55.059694    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f198d8-39a2-4c72-87de-516e077d47bd-tls-certs\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:55.123528 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:55.123454    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj7gn\" (UniqueName: \"kubernetes.io/projected/a5f198d8-39a2-4c72-87de-516e077d47bd-kube-api-access-tj7gn\") pod \"premium-w6-412ba5-kserve-6ffcbf948c-5n46l\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") " pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:55.163540 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:55.163504    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:44:55.535625 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:44:55.535583    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5f198d8_39a2_4c72_87de_516e077d47bd.slice/crio-13a2cb63d4ef4b272d4533ca7ffb290ee70d5cfa34c8f936ef251dcde505ddba WatchSource:0}: Error finding container 13a2cb63d4ef4b272d4533ca7ffb290ee70d5cfa34c8f936ef251dcde505ddba: Status 404 returned error can't find the container with id 13a2cb63d4ef4b272d4533ca7ffb290ee70d5cfa34c8f936ef251dcde505ddba
Oct 31 03:44:55.539240 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:55.539218    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"]
Oct 31 03:44:55.567600 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:55.567573    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l" event={"ID":"a5f198d8-39a2-4c72-87de-516e077d47bd","Type":"ContainerStarted","Data":"13a2cb63d4ef4b272d4533ca7ffb290ee70d5cfa34c8f936ef251dcde505ddba"}
Oct 31 03:44:56.573624 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:56.573593    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l" event={"ID":"a5f198d8-39a2-4c72-87de-516e077d47bd","Type":"ContainerStarted","Data":"adcbc79f85d386554141f2ff73561735fb0ac52fb6fa1da2ea1d1fe90d4337a8"}
Oct 31 03:44:56.694011 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:56.693913    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l" podStartSLOduration=2.693896685 podStartE2EDuration="2.693896685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:44:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:44:56.692806516 +0000 UTC m=+993.278346760" watchObservedRunningTime="2026-10-31 03:44:56.693896685 +0000 UTC m=+993.279436948"
Oct 31 03:44:57.164076 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:44:57.164039    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:45:00.048990 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:00.048949    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:45:00.061661 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:00.061635    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:45:00.604413 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:00.604385    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:45:05.163813 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:05.163785    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:45:05.176337 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:05.176314    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:45:05.313893 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:45:05.313856    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:45:05.627606 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:05.627580    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:45:06.780831 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:06.780802    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"]
Oct 31 03:45:06.784849 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:06.784831    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:06.838072 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:06.838039    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"]
Oct 31 03:45:06.887780 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:06.887753    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-d7b7bc\"/\"kube-root-ca.crt\""
Oct 31 03:45:06.887955 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:06.887860    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-d7b7bc\"/\"default-dockercfg-dfk9z\""
Oct 31 03:45:06.890531 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:06.890515    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-d7b7bc\"/\"openshift-service-ca.crt\""
Oct 31 03:45:06.890589 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:06.890572    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-d7b7bc\"/\"simulator-w1-d7b7bc-kserve-self-signed-certs\""
Oct 31 03:45:06.974008 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:06.973974    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-tmp-dir\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:06.974192 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:06.974117    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-home\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:06.974192 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:06.974161    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wvtz\" (UniqueName: \"kubernetes.io/projected/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-kube-api-access-7wvtz\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:06.974301 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:06.974213    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-dshm\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:06.974414 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:06.974397    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-model-cache\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:06.974449 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:06.974429    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-tls-certs\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:07.075426 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:07.075329    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-tmp-dir\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:07.075603 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:07.075439    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-home\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:07.075603 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:07.075490    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wvtz\" (UniqueName: \"kubernetes.io/projected/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-kube-api-access-7wvtz\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:07.075603 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:07.075528    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-dshm\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:07.075780 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:07.075633    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-model-cache\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:07.075780 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:07.075665    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-tls-certs\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:07.075882 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:07.075851    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-tmp-dir\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:07.076124 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:07.076102    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-home\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:07.076498 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:07.076474    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-model-cache\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:07.078847 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:07.078822    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-dshm\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:07.079034 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:07.079013    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-tls-certs\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:07.138349 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:07.138311    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wvtz\" (UniqueName: \"kubernetes.io/projected/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-kube-api-access-7wvtz\") pod \"simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") " pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:07.397872 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:07.397784    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:07.765216 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:45:07.765176    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfde9e7f9_5a81_4c77_b5ee_7996f5a6a48f.slice/crio-cec4fac8a97f34987c077df63de100a987582680aa1790c1874ffd71302d7a28 WatchSource:0}: Error finding container cec4fac8a97f34987c077df63de100a987582680aa1790c1874ffd71302d7a28: Status 404 returned error can't find the container with id cec4fac8a97f34987c077df63de100a987582680aa1790c1874ffd71302d7a28
Oct 31 03:45:07.767801 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:07.767777    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"]
Oct 31 03:45:08.632615 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:08.632577    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" event={"ID":"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f","Type":"ContainerStarted","Data":"47f4c01c7633c2700a9689e3df6b6f6c639ed7429a310dfd3fc4b3933b92b93e"}
Oct 31 03:45:08.633049 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:08.632624    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" event={"ID":"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f","Type":"ContainerStarted","Data":"cec4fac8a97f34987c077df63de100a987582680aa1790c1874ffd71302d7a28"}
Oct 31 03:45:08.752989 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:08.752927    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" podStartSLOduration=2.75291269 podStartE2EDuration="2.75291269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:45:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:45:08.751671739 +0000 UTC m=+1005.337212001" watchObservedRunningTime="2026-10-31 03:45:08.75291269 +0000 UTC m=+1005.338452986"
Oct 31 03:45:09.398818 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:09.398782    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:17.398713 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:17.398677    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:17.411696 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:17.411671    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:17.685317 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:45:17.685215    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:45:35.313563 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:45:35.313525    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:46:05.312231 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:46:05.312187    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:46:08.062122 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.062083    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"]
Oct 31 03:46:08.065994 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.065973    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.165123 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.165097    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-ae8e27\"/\"premium-w0-ae8e27-kserve-self-signed-certs\""
Oct 31 03:46:08.166617 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.166594    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ade2ace8-0217-404c-9592-85576f16b085-tls-certs\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.166708 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.166643    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-model-cache\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.166818 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.166798    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-789dv\" (UniqueName: \"kubernetes.io/projected/ade2ace8-0217-404c-9592-85576f16b085-kube-api-access-789dv\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.166854 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.166831    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-tmp-dir\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.166925 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.166910    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-home\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.166963 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.166935    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-dshm\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.167214 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.167195    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"]
Oct 31 03:46:08.268029 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.267996    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ade2ace8-0217-404c-9592-85576f16b085-tls-certs\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.268207 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.268054    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-model-cache\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.268207 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.268112    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-789dv\" (UniqueName: \"kubernetes.io/projected/ade2ace8-0217-404c-9592-85576f16b085-kube-api-access-789dv\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.268207 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.268136    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-tmp-dir\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.268207 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.268191    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-home\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.268423 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.268215    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-dshm\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.268648 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.268623    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-model-cache\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.268648 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.268649    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-tmp-dir\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.268796 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.268719    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-home\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.270547 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.270526    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-dshm\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.270743 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.270725    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ade2ace8-0217-404c-9592-85576f16b085-tls-certs\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.355839 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.355762    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-789dv\" (UniqueName: \"kubernetes.io/projected/ade2ace8-0217-404c-9592-85576f16b085-kube-api-access-789dv\") pod \"premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") " pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.377602 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.377560    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:08.756106 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.756080    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"]
Oct 31 03:46:08.759832 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:46:08.759795    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podade2ace8_0217_404c_9592_85576f16b085.slice/crio-40d498f1d2049b2a22e7ac92e03fa0220fb319ca15ae39ebe19a464fc17d3fb9 WatchSource:0}: Error finding container 40d498f1d2049b2a22e7ac92e03fa0220fb319ca15ae39ebe19a464fc17d3fb9: Status 404 returned error can't find the container with id 40d498f1d2049b2a22e7ac92e03fa0220fb319ca15ae39ebe19a464fc17d3fb9
Oct 31 03:46:08.915977 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.915938    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" event={"ID":"ade2ace8-0217-404c-9592-85576f16b085","Type":"ContainerStarted","Data":"0889eadb61cbb22dce35033ee7e8bf29945c2475191c2d928a954d9054733969"}
Oct 31 03:46:08.916123 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:08.915988    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" event={"ID":"ade2ace8-0217-404c-9592-85576f16b085","Type":"ContainerStarted","Data":"40d498f1d2049b2a22e7ac92e03fa0220fb319ca15ae39ebe19a464fc17d3fb9"}
Oct 31 03:46:09.028001 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:09.027902    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" podStartSLOduration=1.027887767 podStartE2EDuration="1.027887767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:46:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:46:09.026970491 +0000 UTC m=+1065.612510754" watchObservedRunningTime="2026-10-31 03:46:09.027887767 +0000 UTC m=+1065.613428028"
Oct 31 03:46:09.378435 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:09.378356    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:18.378571 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:18.378536    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:18.390658 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:18.390631    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:18.972073 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:18.972045    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:46:20.021144 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:20.021110    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"]
Oct 31 03:46:20.021604 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:20.021307    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r" podUID="4dab396a-6307-46dc-9251-52e4dbc7f289" containerName="istio-proxy" containerID="cri-o://ba7d26a0fe5df3e1ae8ed47872ab8d5a57361185db8be00dc87b7085388710ce" gracePeriod=30
Oct 31 03:46:25.277149 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.277120    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:46:25.342164 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.342082    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-data\") pod \"4dab396a-6307-46dc-9251-52e4dbc7f289\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") "
Oct 31 03:46:25.342164 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.342130    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4dab396a-6307-46dc-9251-52e4dbc7f289-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dab396a-6307-46dc-9251-52e4dbc7f289-istiod-ca-cert\") pod \"4dab396a-6307-46dc-9251-52e4dbc7f289\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") "
Oct 31 03:46:25.342164 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.342159    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-token\" (UniqueName: \"kubernetes.io/projected/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-token\") pod \"4dab396a-6307-46dc-9251-52e4dbc7f289\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") "
Oct 31 03:46:25.342422 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.342353    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4dab396a-6307-46dc-9251-52e4dbc7f289-kube-api-access-jgdp5\" (UniqueName: \"kubernetes.io/projected/4dab396a-6307-46dc-9251-52e4dbc7f289-kube-api-access-jgdp5\") pod \"4dab396a-6307-46dc-9251-52e4dbc7f289\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") "
Oct 31 03:46:25.342487 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.342436    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-data" pod "4dab396a-6307-46dc-9251-52e4dbc7f289" (UID: "4dab396a-6307-46dc-9251-52e4dbc7f289"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:46:25.342540 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.342492    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-envoy\") pod \"4dab396a-6307-46dc-9251-52e4dbc7f289\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") "
Oct 31 03:46:25.342540 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.342522    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-credential-socket\") pod \"4dab396a-6307-46dc-9251-52e4dbc7f289\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") "
Oct 31 03:46:25.342653 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.342547    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-workload-socket\") pod \"4dab396a-6307-46dc-9251-52e4dbc7f289\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") "
Oct 31 03:46:25.342653 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.342578    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-podinfo\") pod \"4dab396a-6307-46dc-9251-52e4dbc7f289\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") "
Oct 31 03:46:25.342653 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.342575    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dab396a-6307-46dc-9251-52e4dbc7f289-istiod-ca-cert" pod "4dab396a-6307-46dc-9251-52e4dbc7f289" (UID: "4dab396a-6307-46dc-9251-52e4dbc7f289"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:46:25.342857 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.342813    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-credential-socket" pod "4dab396a-6307-46dc-9251-52e4dbc7f289" (UID: "4dab396a-6307-46dc-9251-52e4dbc7f289"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:46:25.342914 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.342876    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-workload-socket" pod "4dab396a-6307-46dc-9251-52e4dbc7f289" (UID: "4dab396a-6307-46dc-9251-52e4dbc7f289"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:46:25.343077 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.343028    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dab396a-6307-46dc-9251-52e4dbc7f289-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.343077 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.343056    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.343077 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.343072    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.343077 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.343086    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.344688 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.344658    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-token" pod "4dab396a-6307-46dc-9251-52e4dbc7f289" (UID: "4dab396a-6307-46dc-9251-52e4dbc7f289"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:46:25.344688 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.344664    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dab396a-6307-46dc-9251-52e4dbc7f289-kube-api-access-jgdp5" pod "4dab396a-6307-46dc-9251-52e4dbc7f289" (UID: "4dab396a-6307-46dc-9251-52e4dbc7f289"). InnerVolumeSpecName "kube-api-access-jgdp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:46:25.344833 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.344749    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-envoy" pod "4dab396a-6307-46dc-9251-52e4dbc7f289" (UID: "4dab396a-6307-46dc-9251-52e4dbc7f289"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:46:25.344872 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.344841    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-podinfo" pod "4dab396a-6307-46dc-9251-52e4dbc7f289" (UID: "4dab396a-6307-46dc-9251-52e4dbc7f289"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 03:46:25.444253 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.444223    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-workload-certs\") pod \"4dab396a-6307-46dc-9251-52e4dbc7f289\" (UID: \"4dab396a-6307-46dc-9251-52e4dbc7f289\") "
Oct 31 03:46:25.444475 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.444461    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.444527 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.444478    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgdp5\" (UniqueName: \"kubernetes.io/projected/4dab396a-6307-46dc-9251-52e4dbc7f289-kube-api-access-jgdp5\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.444527 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.444489    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.444527 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.444484    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-workload-certs" pod "4dab396a-6307-46dc-9251-52e4dbc7f289" (UID: "4dab396a-6307-46dc-9251-52e4dbc7f289"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:46:25.444527 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.444499    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dab396a-6307-46dc-9251-52e4dbc7f289-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.545142 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.545109    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dab396a-6307-46dc-9251-52e4dbc7f289-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:46:25.997135 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.997101    4926 generic.go:356] "Generic (PLEG): container finished" podID="4dab396a-6307-46dc-9251-52e4dbc7f289" containerID="ba7d26a0fe5df3e1ae8ed47872ab8d5a57361185db8be00dc87b7085388710ce" exitCode=0
Oct 31 03:46:25.997362 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.997149    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"
Oct 31 03:46:25.997362 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.997190    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r" event={"ID":"4dab396a-6307-46dc-9251-52e4dbc7f289","Type":"ContainerDied","Data":"ba7d26a0fe5df3e1ae8ed47872ab8d5a57361185db8be00dc87b7085388710ce"}
Oct 31 03:46:25.997362 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.997260    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r" event={"ID":"4dab396a-6307-46dc-9251-52e4dbc7f289","Type":"ContainerDied","Data":"947b386e255cfa99cbce7ef345d25926b6a8715e1d6c541b71bf421d0a700332"}
Oct 31 03:46:25.997362 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:25.997293    4926 scope.go:122] "RemoveContainer" containerID="ba7d26a0fe5df3e1ae8ed47872ab8d5a57361185db8be00dc87b7085388710ce"
Oct 31 03:46:26.006589 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:26.006569    4926 scope.go:122] "RemoveContainer" containerID="ba7d26a0fe5df3e1ae8ed47872ab8d5a57361185db8be00dc87b7085388710ce"
Oct 31 03:46:26.006891 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:46:26.006869    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba7d26a0fe5df3e1ae8ed47872ab8d5a57361185db8be00dc87b7085388710ce\": container with ID starting with ba7d26a0fe5df3e1ae8ed47872ab8d5a57361185db8be00dc87b7085388710ce not found: ID does not exist" containerID="ba7d26a0fe5df3e1ae8ed47872ab8d5a57361185db8be00dc87b7085388710ce"
Oct 31 03:46:26.006931 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:26.006904    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7d26a0fe5df3e1ae8ed47872ab8d5a57361185db8be00dc87b7085388710ce"} err="failed to get container status \"ba7d26a0fe5df3e1ae8ed47872ab8d5a57361185db8be00dc87b7085388710ce\": rpc error: code = NotFound desc = could not find container \"ba7d26a0fe5df3e1ae8ed47872ab8d5a57361185db8be00dc87b7085388710ce\": container with ID starting with ba7d26a0fe5df3e1ae8ed47872ab8d5a57361185db8be00dc87b7085388710ce not found: ID does not exist"
Oct 31 03:46:26.201864 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:26.201833    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"]
Oct 31 03:46:26.249297 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:26.249198    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-c6k9r"]
Oct 31 03:46:27.538213 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:27.538180    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4dab396a-6307-46dc-9251-52e4dbc7f289" path="/var/lib/kubelet/pods/4dab396a-6307-46dc-9251-52e4dbc7f289/volumes"
Oct 31 03:46:35.314070 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:46:35.314024    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:46:43.044883 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.044843    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"]
Oct 31 03:46:43.045357 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.045235    4926 cpu_manager.go:414] "containerMap: removing container" podUID="4dab396a-6307-46dc-9251-52e4dbc7f289" containerName="istio-proxy"
Oct 31 03:46:43.045357 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.045246    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4dab396a-6307-46dc-9251-52e4dbc7f289" containerName="istio-proxy"
Oct 31 03:46:43.045357 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.045334    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="4dab396a-6307-46dc-9251-52e4dbc7f289" containerName="istio-proxy"
Oct 31 03:46:43.048487 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.048469    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.103092 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.103061    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"]
Oct 31 03:46:43.111418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.111387    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-e5b8f9\"/\"openshift-service-ca.crt\""
Oct 31 03:46:43.119056 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.119026    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20732a1-da5b-4e22-b861-6b03dfe62522-tls-certs\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.119204 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.119061    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-model-cache\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.119204 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.119094    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-tmp-dir\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.119204 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.119158    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r94s\" (UniqueName: \"kubernetes.io/projected/e20732a1-da5b-4e22-b861-6b03dfe62522-kube-api-access-7r94s\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.119386 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.119319    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-dshm\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.119437 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.119381    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-home\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.148243 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.148214    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-e5b8f9\"/\"kube-root-ca.crt\""
Oct 31 03:46:43.148429 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.148306    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-e5b8f9\"/\"premium-w5-e5b8f9-kserve-self-signed-certs\""
Oct 31 03:46:43.148429 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.148370    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-e5b8f9\"/\"default-dockercfg-57f6t\""
Oct 31 03:46:43.220584 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.220549    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20732a1-da5b-4e22-b861-6b03dfe62522-tls-certs\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.220780 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.220591    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-model-cache\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.220780 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.220726    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-tmp-dir\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.220780 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.220767    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r94s\" (UniqueName: \"kubernetes.io/projected/e20732a1-da5b-4e22-b861-6b03dfe62522-kube-api-access-7r94s\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.220947 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.220844    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-dshm\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.220947 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.220893    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-home\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.221049 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.220951    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-model-cache\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.221049 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.221006    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-tmp-dir\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.221258 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.221237    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-home\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.223101 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.223079    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-dshm\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.223237 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.223222    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20732a1-da5b-4e22-b861-6b03dfe62522-tls-certs\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.279949 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.279909    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r94s\" (UniqueName: \"kubernetes.io/projected/e20732a1-da5b-4e22-b861-6b03dfe62522-kube-api-access-7r94s\") pod \"premium-w5-e5b8f9-kserve-df79994f8-54pfn\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") " pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.360134 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.360035    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:43.728855 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:43.728829    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"]
Oct 31 03:46:43.730757 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:46:43.730732    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode20732a1_da5b_4e22_b861_6b03dfe62522.slice/crio-9e5b91e14bebad02fe07ffd0bd4ddafa41b79238695815bfea5a7d0ed35c11a5 WatchSource:0}: Error finding container 9e5b91e14bebad02fe07ffd0bd4ddafa41b79238695815bfea5a7d0ed35c11a5: Status 404 returned error can't find the container with id 9e5b91e14bebad02fe07ffd0bd4ddafa41b79238695815bfea5a7d0ed35c11a5
Oct 31 03:46:44.086218 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:44.086139    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn" event={"ID":"e20732a1-da5b-4e22-b861-6b03dfe62522","Type":"ContainerStarted","Data":"ad0ad05a7736668eb55fb9d5b2847dd9e5220a541c0dfce59698c8723cfe31ed"}
Oct 31 03:46:44.086218 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:44.086179    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn" event={"ID":"e20732a1-da5b-4e22-b861-6b03dfe62522","Type":"ContainerStarted","Data":"9e5b91e14bebad02fe07ffd0bd4ddafa41b79238695815bfea5a7d0ed35c11a5"}
Oct 31 03:46:44.198399 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:44.198348    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn" podStartSLOduration=1.1983341 podStartE2EDuration="1.1983341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:46:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:46:44.197131927 +0000 UTC m=+1100.782672190" watchObservedRunningTime="2026-10-31 03:46:44.1983341 +0000 UTC m=+1100.783874362"
Oct 31 03:46:44.360782 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:44.360702    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:53.360838 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:53.360798    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:53.373622 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:53.373600    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:46:54.144146 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:46:54.144117    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:47:05.313501 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:47:05.313420    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:47:05.448965 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.448933    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"]
Oct 31 03:47:05.452607 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.452584    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.505762 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.505731    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-537e34-openshift-default-dockercfg-5x4wl\""
Oct 31 03:47:05.544842 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.544813    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.545014 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.544880    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.545014 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.544991    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.545155 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.545057    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbtt7\" (UniqueName: \"kubernetes.io/projected/6846468a-e677-43db-82e4-24f28fd7a0f6-kube-api-access-pbtt7\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.545155 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.545092    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.545155 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.545119    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.545250 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.545182    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.545250 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.545216    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.545250 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.545240    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6846468a-e677-43db-82e4-24f28fd7a0f6-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.552400 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.552379    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"]
Oct 31 03:47:05.645939 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.645859    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbtt7\" (UniqueName: \"kubernetes.io/projected/6846468a-e677-43db-82e4-24f28fd7a0f6-kube-api-access-pbtt7\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.645939 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.645898    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.645939 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.645928    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.646232 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.645966    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.646232 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.646010    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.646372 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.646311    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6846468a-e677-43db-82e4-24f28fd7a0f6-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.646435 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.646372    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.646504 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.646487    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.646562 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.646533    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.646833 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.646779    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.646951 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.646875    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.647029 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.646999    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.647130 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.647097    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.647169 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.647136    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6846468a-e677-43db-82e4-24f28fd7a0f6-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.649016 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.648997    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.649246 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.649228    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.704576 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.704548    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.704880 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.704859    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbtt7\" (UniqueName: \"kubernetes.io/projected/6846468a-e677-43db-82e4-24f28fd7a0f6-kube-api-access-pbtt7\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:05.766809 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:05.766778    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:06.091007 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.090968    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"]
Oct 31 03:47:06.100037 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.100008    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.150535 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.150506    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-412ba5-openshift-default-dockercfg-dglp9\""
Oct 31 03:47:06.151497 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.151469    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"]
Oct 31 03:47:06.151621 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.151502    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.151621 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.151542    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.151621 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.151616    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.151760 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.151657    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.151795 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.151762    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.151840 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.151813    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.151880 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.151862    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/136c00fa-e898-4bfd-9b48-9241fba6740e-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.151933 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.151897    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.152001 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.151978    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpbx6\" (UniqueName: \"kubernetes.io/projected/136c00fa-e898-4bfd-9b48-9241fba6740e-kube-api-access-cpbx6\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.242749 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.242722    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"]
Oct 31 03:47:06.244476 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:47:06.244448    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6846468a_e677_43db_82e4_24f28fd7a0f6.slice/crio-dc23eae2528569419ea382de9754a9e4cb6df79e32ed9be3dd593ab546c1b92a WatchSource:0}: Error finding container dc23eae2528569419ea382de9754a9e4cb6df79e32ed9be3dd593ab546c1b92a: Status 404 returned error can't find the container with id dc23eae2528569419ea382de9754a9e4cb6df79e32ed9be3dd593ab546c1b92a
Oct 31 03:47:06.246892 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.246859    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:47:06.246988 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.246939    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:47:06.247050 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.246985    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:47:06.253258 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.253238    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.253337 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.253287    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/136c00fa-e898-4bfd-9b48-9241fba6740e-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.253337 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.253324    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.253417 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.253370    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpbx6\" (UniqueName: \"kubernetes.io/projected/136c00fa-e898-4bfd-9b48-9241fba6740e-kube-api-access-cpbx6\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.253523 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.253499    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.253569 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.253546    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.253620 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.253592    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.253675 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.253634    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.253675 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.253652    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.253794 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.253693    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.253794 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.253716    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.253933 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.253912    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.254005 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.253983    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/136c00fa-e898-4bfd-9b48-9241fba6740e-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.254056 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.254041    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.256020 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.255995    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.256358 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.256338    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.321657 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.321621    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.326497 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.326471    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpbx6\" (UniqueName: \"kubernetes.io/projected/136c00fa-e898-4bfd-9b48-9241fba6740e-kube-api-access-cpbx6\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.413061 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.412985    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:06.834538 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.834504    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"]
Oct 31 03:47:06.838706 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:47:06.838664    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod136c00fa_e898_4bfd_9b48_9241fba6740e.slice/crio-01c3056c858f18f6d7ffe4630f1df413d6241e941e31de671f71d118cd183a45 WatchSource:0}: Error finding container 01c3056c858f18f6d7ffe4630f1df413d6241e941e31de671f71d118cd183a45: Status 404 returned error can't find the container with id 01c3056c858f18f6d7ffe4630f1df413d6241e941e31de671f71d118cd183a45
Oct 31 03:47:06.841103 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.841074    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:47:06.841195 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.841141    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:47:06.841195 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:06.841169    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:47:07.196118 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:07.196023    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp" event={"ID":"6846468a-e677-43db-82e4-24f28fd7a0f6","Type":"ContainerStarted","Data":"6c1759996cbe947bb55870f4252a70a1f6bf8e4be459c4872d39ab8f3c0b0478"}
Oct 31 03:47:07.196118 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:07.196090    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp" event={"ID":"6846468a-e677-43db-82e4-24f28fd7a0f6","Type":"ContainerStarted","Data":"dc23eae2528569419ea382de9754a9e4cb6df79e32ed9be3dd593ab546c1b92a"}
Oct 31 03:47:07.197633 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:07.197608    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm" event={"ID":"136c00fa-e898-4bfd-9b48-9241fba6740e","Type":"ContainerStarted","Data":"d17a6464f2bdb469ae68d3613c098a3cfd60fe322756b2e7772cc6c3b4841a81"}
Oct 31 03:47:07.197734 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:07.197648    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm" event={"ID":"136c00fa-e898-4bfd-9b48-9241fba6740e","Type":"ContainerStarted","Data":"01c3056c858f18f6d7ffe4630f1df413d6241e941e31de671f71d118cd183a45"}
Oct 31 03:47:07.335463 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:07.314356    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp" podStartSLOduration=2.314337326 podStartE2EDuration="2.314337326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:47:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:47:07.313765826 +0000 UTC m=+1123.899306089" watchObservedRunningTime="2026-10-31 03:47:07.314337326 +0000 UTC m=+1123.899877589"
Oct 31 03:47:07.417294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:07.413619    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:07.438293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:07.427413    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Oct 31 03:47:07.438293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:07.427480    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm" podUID="136c00fa-e898-4bfd-9b48-9241fba6740e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Oct 31 03:47:07.444317 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:07.442113    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm" podStartSLOduration=1.442093733 podStartE2EDuration="1.442093733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:47:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:47:07.441808141 +0000 UTC m=+1124.027348404" watchObservedRunningTime="2026-10-31 03:47:07.442093733 +0000 UTC m=+1124.027633996"
Oct 31 03:47:07.767848 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:07.767803    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:07.769636 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:07.769608    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Oct 31 03:47:07.769766 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:07.769664    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp" podUID="6846468a-e677-43db-82e4-24f28fd7a0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Oct 31 03:47:08.426294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:08.424294    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Oct 31 03:47:08.426294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:08.424368    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm" podUID="136c00fa-e898-4bfd-9b48-9241fba6740e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Oct 31 03:47:08.768000 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:08.767956    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Oct 31 03:47:08.768190 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:08.768037    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp" podUID="6846468a-e677-43db-82e4-24f28fd7a0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Oct 31 03:47:09.417291 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:09.416656    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Oct 31 03:47:09.417291 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:09.416732    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm" podUID="136c00fa-e898-4bfd-9b48-9241fba6740e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Oct 31 03:47:09.767836 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:09.767798    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Oct 31 03:47:09.768260 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:09.767855    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp" podUID="6846468a-e677-43db-82e4-24f28fd7a0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Oct 31 03:47:10.417293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:10.416657    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Oct 31 03:47:10.417293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:10.416728    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm" podUID="136c00fa-e898-4bfd-9b48-9241fba6740e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Oct 31 03:47:10.767553 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:10.767517    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Oct 31 03:47:10.767742 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:10.767574    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp" podUID="6846468a-e677-43db-82e4-24f28fd7a0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Oct 31 03:47:11.416294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:11.413640    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Oct 31 03:47:11.416294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:11.413698    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm" podUID="136c00fa-e898-4bfd-9b48-9241fba6740e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Oct 31 03:47:11.767450 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:11.767411    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Oct 31 03:47:11.767738 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:11.767710    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp" podUID="6846468a-e677-43db-82e4-24f28fd7a0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Oct 31 03:47:12.416709 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:12.416669    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Oct 31 03:47:12.417343 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:12.417309    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm" podUID="136c00fa-e898-4bfd-9b48-9241fba6740e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Oct 31 03:47:12.767791 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:12.767752    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Oct 31 03:47:12.767974 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:12.767809    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp" podUID="6846468a-e677-43db-82e4-24f28fd7a0f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Oct 31 03:47:13.693647 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:13.693618    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:13.694108 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:13.693891    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:13.694795 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:13.694773    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 03:47:13.771781 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:13.771752    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:13.772051 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:13.772035    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:13.772750 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:47:13.772732    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 03:47:35.312676 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:47:35.312638    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:48:05.313543 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:48:05.313501    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:48:09.578614 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.578581    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"]
Oct 31 03:48:09.582057 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.582037    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.662629 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.662591    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-home\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.662820 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.662696    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-model-cache\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.662820 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.662788    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-tmp-dir\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.662965 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.662835    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdn8x\" (UniqueName: \"kubernetes.io/projected/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-kube-api-access-rdn8x\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.662965 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.662932    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-dshm\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.662965 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.662955    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-tls-certs\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.679685 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.679654    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-412ba5\"/\"distinct-w6-412ba5-kserve-self-signed-certs\""
Oct 31 03:48:09.682865 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.682846    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"]
Oct 31 03:48:09.763673 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.763638    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-dshm\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.763673 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.763678    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-tls-certs\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.763937 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.763739    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-home\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.763937 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.763780    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-model-cache\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.763937 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.763861    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-tmp-dir\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.763937 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.763900    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdn8x\" (UniqueName: \"kubernetes.io/projected/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-kube-api-access-rdn8x\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.764180 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.764149    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-home\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.764238 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.764192    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-tmp-dir\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.764319 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.764233    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-model-cache\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.766081 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.766059    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-dshm\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.766492 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.766473    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-tls-certs\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.822912 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.822870    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdn8x\" (UniqueName: \"kubernetes.io/projected/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-kube-api-access-rdn8x\") pod \"distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") " pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:09.896389 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:09.896212    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:10.267658 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:10.267632    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"]
Oct 31 03:48:10.269855 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:48:10.269828    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5699efa_62d9_4cb3_b53b_bd83adff2f1d.slice/crio-c2ab9f01aef83a5d0869bbc5b268b43733666ddf1d25620de1fa15024ab77160 WatchSource:0}: Error finding container c2ab9f01aef83a5d0869bbc5b268b43733666ddf1d25620de1fa15024ab77160: Status 404 returned error can't find the container with id c2ab9f01aef83a5d0869bbc5b268b43733666ddf1d25620de1fa15024ab77160
Oct 31 03:48:10.560536 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:10.560443    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" event={"ID":"e5699efa-62d9-4cb3-b53b-bd83adff2f1d","Type":"ContainerStarted","Data":"6e06f93869ac36c4449d03568997c1a54322e218503a8513c9a8bdc3007e47aa"}
Oct 31 03:48:10.560536 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:10.560493    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" event={"ID":"e5699efa-62d9-4cb3-b53b-bd83adff2f1d","Type":"ContainerStarted","Data":"c2ab9f01aef83a5d0869bbc5b268b43733666ddf1d25620de1fa15024ab77160"}
Oct 31 03:48:10.671857 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:10.671800    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" podStartSLOduration=1.671782401 podStartE2EDuration="1.671782401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:48:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:48:10.671449303 +0000 UTC m=+1187.256989565" watchObservedRunningTime="2026-10-31 03:48:10.671782401 +0000 UTC m=+1187.257322663"
Oct 31 03:48:10.896792 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:10.896710    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:19.897167 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:19.897131    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:19.910543 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:19.910517    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:20.620666 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:20.620631    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:48:22.243862 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.243828    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"]
Oct 31 03:48:22.247611 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.247591    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.297980 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.297955    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-a64fd2-openshift-default-dockercfg-2llsl\""
Oct 31 03:48:22.347235 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.347210    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"]
Oct 31 03:48:22.405526 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.405499    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88086a57-3749-41e6-8d6b-0fe95866237e-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.405695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.405552    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88086a57-3749-41e6-8d6b-0fe95866237e-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.405695 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.405621    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z45q5\" (UniqueName: \"kubernetes.io/projected/88086a57-3749-41e6-8d6b-0fe95866237e-kube-api-access-z45q5\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.405784 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.405695    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.405784 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.405714    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.405784 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.405758    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88086a57-3749-41e6-8d6b-0fe95866237e-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.405883 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.405827    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.405883 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.405869    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.405952 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.405905    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.507438 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.507359    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.507438 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.507405    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.507649 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.507444    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88086a57-3749-41e6-8d6b-0fe95866237e-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.507649 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.507493    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88086a57-3749-41e6-8d6b-0fe95866237e-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.507649 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.507521    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z45q5\" (UniqueName: \"kubernetes.io/projected/88086a57-3749-41e6-8d6b-0fe95866237e-kube-api-access-z45q5\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.507649 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.507571    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.507649 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.507595    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.507951 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.507658    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88086a57-3749-41e6-8d6b-0fe95866237e-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.507951 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.507692    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.507951 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.507905    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.508071 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.507944    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.508105 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.508086    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.508179 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.508116    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.508386 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.508369    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88086a57-3749-41e6-8d6b-0fe95866237e-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.510061 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.510040    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.510261 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.510240    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88086a57-3749-41e6-8d6b-0fe95866237e-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.566100 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.566069    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88086a57-3749-41e6-8d6b-0fe95866237e-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.566286 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.566171    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z45q5\" (UniqueName: \"kubernetes.io/projected/88086a57-3749-41e6-8d6b-0fe95866237e-kube-api-access-z45q5\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:22.860513 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:22.860432    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:23.278647 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:23.278621    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"]
Oct 31 03:48:23.279789 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:48:23.279749    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88086a57_3749_41e6_8d6b_0fe95866237e.slice/crio-321002edb7e39a9d540c0cac3f2191e26794feae013061fe497e978639c494ff WatchSource:0}: Error finding container 321002edb7e39a9d540c0cac3f2191e26794feae013061fe497e978639c494ff: Status 404 returned error can't find the container with id 321002edb7e39a9d540c0cac3f2191e26794feae013061fe497e978639c494ff
Oct 31 03:48:23.282177 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:23.282144    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:48:23.282293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:23.282214    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:48:23.282293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:23.282240    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:48:23.634226 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:23.634137    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92" event={"ID":"88086a57-3749-41e6-8d6b-0fe95866237e","Type":"ContainerStarted","Data":"76f8e9a679714ae42f643032203cf655542e664add8fb0b0bb434be47c9cc263"}
Oct 31 03:48:23.634723 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:23.634670    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92" event={"ID":"88086a57-3749-41e6-8d6b-0fe95866237e","Type":"ContainerStarted","Data":"321002edb7e39a9d540c0cac3f2191e26794feae013061fe497e978639c494ff"}
Oct 31 03:48:23.645061 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:23.645031    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 03:48:23.645724 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:23.645703    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 03:48:23.752077 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:23.752015    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92" podStartSLOduration=1.751994282 podStartE2EDuration="1.751994282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:48:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:48:23.749708931 +0000 UTC m=+1200.335249206" watchObservedRunningTime="2026-10-31 03:48:23.751994282 +0000 UTC m=+1200.337534545"
Oct 31 03:48:23.860537 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:23.860500    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:24.862390 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:24.862356    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Oct 31 03:48:24.862876 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:24.862412    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92" podUID="88086a57-3749-41e6-8d6b-0fe95866237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Oct 31 03:48:25.861500 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:25.861465    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Oct 31 03:48:25.861751 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:25.861523    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92" podUID="88086a57-3749-41e6-8d6b-0fe95866237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Oct 31 03:48:26.861520 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:26.861487    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Oct 31 03:48:26.861938 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:26.861566    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92" podUID="88086a57-3749-41e6-8d6b-0fe95866237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Oct 31 03:48:27.861798 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:27.861755    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Oct 31 03:48:27.862237 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:27.861829    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92" podUID="88086a57-3749-41e6-8d6b-0fe95866237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Oct 31 03:48:28.861597 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:28.861551    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Oct 31 03:48:28.861793 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:28.861626    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92" podUID="88086a57-3749-41e6-8d6b-0fe95866237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Oct 31 03:48:29.934741 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:29.934711    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:29.935238 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:29.935124    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:29.935822 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:29.935802    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 03:48:35.313656 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:48:35.313615    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:48:51.887363 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:51.887331    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"]
Oct 31 03:48:51.891144 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:51.891125    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:51.980298 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:51.976609    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"]
Oct 31 03:48:52.034007 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.033961    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.034216 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.034139    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.034216 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.034173    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.034216 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.034198    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.034454 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.034346    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.034454 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.034396    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxvzq\" (UniqueName: \"kubernetes.io/projected/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-kube-api-access-cxvzq\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.034573 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.034462    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.034573 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.034503    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.034573 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.034543    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.136186 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.136144    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.136384 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.136289    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.136384 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.136323    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.136384 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.136349    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.136560 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.136401    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.136560 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.136436    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxvzq\" (UniqueName: \"kubernetes.io/projected/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-kube-api-access-cxvzq\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.136560 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.136475    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.136560 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.136518    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.136773 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.136557    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.136773 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.136570    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.136773 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.136696    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.137224 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.137198    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.137470 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.137201    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.137470 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.137303    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.139613 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.139583    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.139905 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.139883    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.195702 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.195665    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxvzq\" (UniqueName: \"kubernetes.io/projected/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-kube-api-access-cxvzq\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.196036 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.196016    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.204448 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.204430    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:52.646231 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.646202    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"]
Oct 31 03:48:52.648045 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:48:52.648020    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2524a945_cbb0_42b3_b2be_8cd43a1f1df9.slice/crio-c344932a25e3d2a40170cf6c20799365426ea7347158d1d42db9cc21ad2eaa13 WatchSource:0}: Error finding container c344932a25e3d2a40170cf6c20799365426ea7347158d1d42db9cc21ad2eaa13: Status 404 returned error can't find the container with id c344932a25e3d2a40170cf6c20799365426ea7347158d1d42db9cc21ad2eaa13
Oct 31 03:48:52.650441 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.650409    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:48:52.650531 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.650474    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:48:52.650531 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.650504    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:48:52.787620 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.787584    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj" event={"ID":"2524a945-cbb0-42b3-b2be-8cd43a1f1df9","Type":"ContainerStarted","Data":"cb6965d4a09e2cb6ebb2ed4108846014f4a0ae2b94048100b9761b88bdf7939e"}
Oct 31 03:48:52.787802 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.787643    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj" event={"ID":"2524a945-cbb0-42b3-b2be-8cd43a1f1df9","Type":"ContainerStarted","Data":"c344932a25e3d2a40170cf6c20799365426ea7347158d1d42db9cc21ad2eaa13"}
Oct 31 03:48:52.904220 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:52.904118    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj" podStartSLOduration=1.904104716 podStartE2EDuration="1.904104716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:48:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:48:52.902175168 +0000 UTC m=+1229.487715429" watchObservedRunningTime="2026-10-31 03:48:52.904104716 +0000 UTC m=+1229.489644977"
Oct 31 03:48:53.204974 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:53.204939    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:48:53.206526 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:53.206505    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Oct 31 03:48:53.206583 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:53.206550    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj" podUID="2524a945-cbb0-42b3-b2be-8cd43a1f1df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Oct 31 03:48:54.205711 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:54.205671    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Oct 31 03:48:54.206110 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:54.205724    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj" podUID="2524a945-cbb0-42b3-b2be-8cd43a1f1df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Oct 31 03:48:55.205609 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:55.205572    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Oct 31 03:48:55.205787 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:55.205626    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj" podUID="2524a945-cbb0-42b3-b2be-8cd43a1f1df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Oct 31 03:48:56.205007 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:56.204968    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Oct 31 03:48:56.205192 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:56.205039    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj" podUID="2524a945-cbb0-42b3-b2be-8cd43a1f1df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Oct 31 03:48:57.205622 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:57.205583    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Oct 31 03:48:57.206119 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:57.205635    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj" podUID="2524a945-cbb0-42b3-b2be-8cd43a1f1df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Oct 31 03:48:58.205448 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:58.205408    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Oct 31 03:48:58.205611 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:58.205475    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj" podUID="2524a945-cbb0-42b3-b2be-8cd43a1f1df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Oct 31 03:48:59.205857 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:59.205815    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Oct 31 03:48:59.206405 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:48:59.205917    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj" podUID="2524a945-cbb0-42b3-b2be-8cd43a1f1df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Oct 31 03:49:00.210434 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:00.210402    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:49:00.210897 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:00.210736    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:49:00.211664 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:00.211646    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 03:49:05.315542 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:49:05.315506    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:49:22.474360 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.474327    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"]
Oct 31 03:49:22.477963 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.477946    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.529190 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.529160    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-e5b8f9-openshift-default-dockercfg-qt6w5\""
Oct 31 03:49:22.559253 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.559226    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.559398 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.559369    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/362761bb-e08f-4109-8989-3acebfc6b762-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.559458 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.559441    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt6sm\" (UniqueName: \"kubernetes.io/projected/362761bb-e08f-4109-8989-3acebfc6b762-kube-api-access-qt6sm\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.559545 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.559521    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.559604 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.559592    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.559676 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.559662    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.559799 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.559784    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/362761bb-e08f-4109-8989-3acebfc6b762-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.559842 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.559817    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/362761bb-e08f-4109-8989-3acebfc6b762-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.559896 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.559880    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.579974 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.579946    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"]
Oct 31 03:49:22.660644 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.660606    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.660644 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.660643    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/362761bb-e08f-4109-8989-3acebfc6b762-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.660870 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.660670    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/362761bb-e08f-4109-8989-3acebfc6b762-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.660870 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.660703    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.660870 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.660748    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.660870 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.660811    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/362761bb-e08f-4109-8989-3acebfc6b762-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.660870 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.660856    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt6sm\" (UniqueName: \"kubernetes.io/projected/362761bb-e08f-4109-8989-3acebfc6b762-kube-api-access-qt6sm\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.661124 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.660897    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.661124 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.660966    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.661682 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.661648    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.661862 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.661699    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.662031 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.661980    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.662153 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.662028    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.663303 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.662289    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/362761bb-e08f-4109-8989-3acebfc6b762-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.663473 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.663451    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.672814 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.672792    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/362761bb-e08f-4109-8989-3acebfc6b762-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.719153 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.719125    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/362761bb-e08f-4109-8989-3acebfc6b762-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.719767 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.719742    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt6sm\" (UniqueName: \"kubernetes.io/projected/362761bb-e08f-4109-8989-3acebfc6b762-kube-api-access-qt6sm\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:22.791758 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:22.791686    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:23.211904 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:23.211882    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"]
Oct 31 03:49:23.213023 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:49:23.212998    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod362761bb_e08f_4109_8989_3acebfc6b762.slice/crio-a2f67786d1249e92a49a4ac9d7544514abbdb75a356238b11219c384d2beacef WatchSource:0}: Error finding container a2f67786d1249e92a49a4ac9d7544514abbdb75a356238b11219c384d2beacef: Status 404 returned error can't find the container with id a2f67786d1249e92a49a4ac9d7544514abbdb75a356238b11219c384d2beacef
Oct 31 03:49:23.215300 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:23.215239    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:49:23.215411 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:23.215330    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:49:23.215411 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:23.215375    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:49:23.955178 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:23.955141    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv" event={"ID":"362761bb-e08f-4109-8989-3acebfc6b762","Type":"ContainerStarted","Data":"45124883a5690f73fe168243314dfaf40f58ed2e28129b43f0d165e7c8835870"}
Oct 31 03:49:23.955707 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:23.955205    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv" event={"ID":"362761bb-e08f-4109-8989-3acebfc6b762","Type":"ContainerStarted","Data":"a2f67786d1249e92a49a4ac9d7544514abbdb75a356238b11219c384d2beacef"}
Oct 31 03:49:24.792186 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:24.792158    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:24.793513 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:24.793481    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 31 03:49:24.793663 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:24.793535    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv" podUID="362761bb-e08f-4109-8989-3acebfc6b762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 31 03:49:25.792464 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:25.792429    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 31 03:49:25.792910 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:25.792484    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv" podUID="362761bb-e08f-4109-8989-3acebfc6b762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 31 03:49:26.792311 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:26.792279    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 31 03:49:26.792485 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:26.792344    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv" podUID="362761bb-e08f-4109-8989-3acebfc6b762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 31 03:49:27.792793 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:27.792759    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 31 03:49:27.793187 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:27.792817    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv" podUID="362761bb-e08f-4109-8989-3acebfc6b762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 31 03:49:28.792006 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:28.791968    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 31 03:49:28.792355 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:28.792038    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv" podUID="362761bb-e08f-4109-8989-3acebfc6b762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 31 03:49:29.792543 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:29.792500    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Oct 31 03:49:29.792935 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:29.792592    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv" podUID="362761bb-e08f-4109-8989-3acebfc6b762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Oct 31 03:49:30.796900 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:30.796869    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:30.797383 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:30.797132    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:30.797866 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:30.797848    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 03:49:30.961141 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:30.961085    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv" podStartSLOduration=8.961071972 podStartE2EDuration="8.961071972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:49:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:49:24.126762485 +0000 UTC m=+1260.712302742" watchObservedRunningTime="2026-10-31 03:49:30.961071972 +0000 UTC m=+1267.546612234"
Oct 31 03:49:35.312449 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:49:35.312400    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:49:36.461257 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:36.461221    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"]
Oct 31 03:49:36.461779 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:36.461428    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl" podUID="637fff8e-91cf-44c3-b626-def342c072c9" containerName="istio-proxy" containerID="cri-o://026c1ca7f4e86df69857ca84bc4f1f114c5980b91e7296f978511f982cb5f39c" gracePeriod=30
Oct 31 03:49:41.325419 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.325382    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-md4xl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 03:49:41.325800 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.325451    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl" podUID="637fff8e-91cf-44c3-b626-def342c072c9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 03:49:41.748712 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.748687    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:49:41.790173 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.790137    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/637fff8e-91cf-44c3-b626-def342c072c9-istio-token\" (UniqueName: \"kubernetes.io/projected/637fff8e-91cf-44c3-b626-def342c072c9-istio-token\") pod \"637fff8e-91cf-44c3-b626-def342c072c9\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") "
Oct 31 03:49:41.790375 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.790182    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-credential-socket\") pod \"637fff8e-91cf-44c3-b626-def342c072c9\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") "
Oct 31 03:49:41.790375 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.790250    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-workload-socket\") pod \"637fff8e-91cf-44c3-b626-def342c072c9\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") "
Oct 31 03:49:41.790375 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.790341    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/637fff8e-91cf-44c3-b626-def342c072c9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/637fff8e-91cf-44c3-b626-def342c072c9-istio-podinfo\") pod \"637fff8e-91cf-44c3-b626-def342c072c9\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") "
Oct 31 03:49:41.790554 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.790387    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-workload-certs\") pod \"637fff8e-91cf-44c3-b626-def342c072c9\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") "
Oct 31 03:49:41.790554 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.790415    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-istio-data\") pod \"637fff8e-91cf-44c3-b626-def342c072c9\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") "
Oct 31 03:49:41.790554 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.790448    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/637fff8e-91cf-44c3-b626-def342c072c9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/637fff8e-91cf-44c3-b626-def342c072c9-istiod-ca-cert\") pod \"637fff8e-91cf-44c3-b626-def342c072c9\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") "
Oct 31 03:49:41.790554 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.790514    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-istio-envoy\") pod \"637fff8e-91cf-44c3-b626-def342c072c9\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") "
Oct 31 03:49:41.790554 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.790536    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-credential-socket" pod "637fff8e-91cf-44c3-b626-def342c072c9" (UID: "637fff8e-91cf-44c3-b626-def342c072c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:49:41.790827 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.790568    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/637fff8e-91cf-44c3-b626-def342c072c9-kube-api-access-djwh4\" (UniqueName: \"kubernetes.io/projected/637fff8e-91cf-44c3-b626-def342c072c9-kube-api-access-djwh4\") pod \"637fff8e-91cf-44c3-b626-def342c072c9\" (UID: \"637fff8e-91cf-44c3-b626-def342c072c9\") "
Oct 31 03:49:41.790827 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.790709    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-workload-certs" pod "637fff8e-91cf-44c3-b626-def342c072c9" (UID: "637fff8e-91cf-44c3-b626-def342c072c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:49:41.790827 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.790727    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-istio-data" pod "637fff8e-91cf-44c3-b626-def342c072c9" (UID: "637fff8e-91cf-44c3-b626-def342c072c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:49:41.790991 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.790849    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-workload-socket" pod "637fff8e-91cf-44c3-b626-def342c072c9" (UID: "637fff8e-91cf-44c3-b626-def342c072c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:49:41.791098 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.791077    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:49:41.791134 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.791106    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:49:41.791134 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.791121    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:49:41.791222 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.791135    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:49:41.791393 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.791373    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/637fff8e-91cf-44c3-b626-def342c072c9-istiod-ca-cert" pod "637fff8e-91cf-44c3-b626-def342c072c9" (UID: "637fff8e-91cf-44c3-b626-def342c072c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:49:41.792895 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.792865    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/637fff8e-91cf-44c3-b626-def342c072c9-istio-token" pod "637fff8e-91cf-44c3-b626-def342c072c9" (UID: "637fff8e-91cf-44c3-b626-def342c072c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:49:41.793171 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.793144    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-istio-envoy" pod "637fff8e-91cf-44c3-b626-def342c072c9" (UID: "637fff8e-91cf-44c3-b626-def342c072c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:49:41.793370 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.793350    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/637fff8e-91cf-44c3-b626-def342c072c9-kube-api-access-djwh4" pod "637fff8e-91cf-44c3-b626-def342c072c9" (UID: "637fff8e-91cf-44c3-b626-def342c072c9"). InnerVolumeSpecName "kube-api-access-djwh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:49:41.793685 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.793665    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/637fff8e-91cf-44c3-b626-def342c072c9-istio-podinfo" pod "637fff8e-91cf-44c3-b626-def342c072c9" (UID: "637fff8e-91cf-44c3-b626-def342c072c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 03:49:41.891957 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.891877    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/637fff8e-91cf-44c3-b626-def342c072c9-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:49:41.891957 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.891904    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/637fff8e-91cf-44c3-b626-def342c072c9-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:49:41.891957 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.891915    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/637fff8e-91cf-44c3-b626-def342c072c9-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:49:41.891957 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.891926    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/637fff8e-91cf-44c3-b626-def342c072c9-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:49:41.891957 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:41.891937    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djwh4\" (UniqueName: \"kubernetes.io/projected/637fff8e-91cf-44c3-b626-def342c072c9-kube-api-access-djwh4\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:49:42.057467 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:42.057427    4926 generic.go:356] "Generic (PLEG): container finished" podID="637fff8e-91cf-44c3-b626-def342c072c9" containerID="026c1ca7f4e86df69857ca84bc4f1f114c5980b91e7296f978511f982cb5f39c" exitCode=0
Oct 31 03:49:42.057662 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:42.057467    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl" event={"ID":"637fff8e-91cf-44c3-b626-def342c072c9","Type":"ContainerDied","Data":"026c1ca7f4e86df69857ca84bc4f1f114c5980b91e7296f978511f982cb5f39c"}
Oct 31 03:49:42.057662 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:42.057499    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"
Oct 31 03:49:42.057662 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:42.057556    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl" event={"ID":"637fff8e-91cf-44c3-b626-def342c072c9","Type":"ContainerDied","Data":"1f0c46fd7f51f38312a8b8186630245b4ce4f7739d1bbde05b1adc8a52e9dbf9"}
Oct 31 03:49:42.057662 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:42.057578    4926 scope.go:122] "RemoveContainer" containerID="026c1ca7f4e86df69857ca84bc4f1f114c5980b91e7296f978511f982cb5f39c"
Oct 31 03:49:42.071485 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:42.071463    4926 scope.go:122] "RemoveContainer" containerID="026c1ca7f4e86df69857ca84bc4f1f114c5980b91e7296f978511f982cb5f39c"
Oct 31 03:49:42.071745 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:49:42.071726    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"026c1ca7f4e86df69857ca84bc4f1f114c5980b91e7296f978511f982cb5f39c\": container with ID starting with 026c1ca7f4e86df69857ca84bc4f1f114c5980b91e7296f978511f982cb5f39c not found: ID does not exist" containerID="026c1ca7f4e86df69857ca84bc4f1f114c5980b91e7296f978511f982cb5f39c"
Oct 31 03:49:42.071800 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:42.071753    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"026c1ca7f4e86df69857ca84bc4f1f114c5980b91e7296f978511f982cb5f39c"} err="failed to get container status \"026c1ca7f4e86df69857ca84bc4f1f114c5980b91e7296f978511f982cb5f39c\": rpc error: code = NotFound desc = could not find container \"026c1ca7f4e86df69857ca84bc4f1f114c5980b91e7296f978511f982cb5f39c\": container with ID starting with 026c1ca7f4e86df69857ca84bc4f1f114c5980b91e7296f978511f982cb5f39c not found: ID does not exist"
Oct 31 03:49:42.216214 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:42.216179    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"]
Oct 31 03:49:42.219996 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:42.219973    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-md4xl"]
Oct 31 03:49:43.265184 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.265152    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"]
Oct 31 03:49:43.265693 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.265676    4926 cpu_manager.go:414] "containerMap: removing container" podUID="637fff8e-91cf-44c3-b626-def342c072c9" containerName="istio-proxy"
Oct 31 03:49:43.265742 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.265695    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="637fff8e-91cf-44c3-b626-def342c072c9" containerName="istio-proxy"
Oct 31 03:49:43.265779 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.265773    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="637fff8e-91cf-44c3-b626-def342c072c9" containerName="istio-proxy"
Oct 31 03:49:43.270060 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.270042    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.307706 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.307653    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpcxg\" (UniqueName: \"kubernetes.io/projected/3f4959e7-6fb3-4687-8d65-664ef92519f5-kube-api-access-xpcxg\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.307881 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.307714    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-tmp-dir\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.307881 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.307804    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-model-cache\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.307994 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.307900    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-home\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.308051 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.308025    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f4959e7-6fb3-4687-8d65-664ef92519f5-tls-certs\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.308106 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.308087    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-dshm\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.323109 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.323079    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"]
Oct 31 03:49:43.367726 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.367699    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-537e34\"/\"distinct-w3-537e34-kserve-self-signed-certs\""
Oct 31 03:49:43.409861 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.409384    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-home\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.409861 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.409442    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f4959e7-6fb3-4687-8d65-664ef92519f5-tls-certs\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.409861 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.409533    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-dshm\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.409861 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.409593    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpcxg\" (UniqueName: \"kubernetes.io/projected/3f4959e7-6fb3-4687-8d65-664ef92519f5-kube-api-access-xpcxg\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.409861 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.409633    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-tmp-dir\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.409861 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.409664    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-model-cache\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.410441 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.410342    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-model-cache\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.410761 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.410737    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-tmp-dir\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.411477 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.411452    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-home\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.414930 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.414905    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-dshm\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.415117 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.415097    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f4959e7-6fb3-4687-8d65-664ef92519f5-tls-certs\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.469797 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.469758    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpcxg\" (UniqueName: \"kubernetes.io/projected/3f4959e7-6fb3-4687-8d65-664ef92519f5-kube-api-access-xpcxg\") pod \"distinct-w3-537e34-kserve-69b646b9bf-xjbwr\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.538773 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.538667    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="637fff8e-91cf-44c3-b626-def342c072c9" path="/var/lib/kubelet/pods/637fff8e-91cf-44c3-b626-def342c072c9/volumes"
Oct 31 03:49:43.581036 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.580997    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:43.987828 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:43.987800    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"]
Oct 31 03:49:43.988822 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:49:43.988796    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3f4959e7_6fb3_4687_8d65_664ef92519f5.slice/crio-071d0eb419b25a65f054e159f3340de7eb6c63011159a1207d6e0eaa31e25296 WatchSource:0}: Error finding container 071d0eb419b25a65f054e159f3340de7eb6c63011159a1207d6e0eaa31e25296: Status 404 returned error can't find the container with id 071d0eb419b25a65f054e159f3340de7eb6c63011159a1207d6e0eaa31e25296
Oct 31 03:49:44.071816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:44.071779    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" event={"ID":"3f4959e7-6fb3-4687-8d65-664ef92519f5","Type":"ContainerStarted","Data":"60328cd17c6eb4f894f4458ea1c38118fcfaf7e67d0f928c6103dce9a05fbfe6"}
Oct 31 03:49:44.071960 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:44.071832    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" event={"ID":"3f4959e7-6fb3-4687-8d65-664ef92519f5","Type":"ContainerStarted","Data":"071d0eb419b25a65f054e159f3340de7eb6c63011159a1207d6e0eaa31e25296"}
Oct 31 03:49:44.184232 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:44.184177    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" podStartSLOduration=1.1841621390000001 podStartE2EDuration="1.184162139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:49:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:49:44.183171964 +0000 UTC m=+1280.768712226" watchObservedRunningTime="2026-10-31 03:49:44.184162139 +0000 UTC m=+1280.769702398"
Oct 31 03:49:44.581655 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:44.581624    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:52.571748 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.571714    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"]
Oct 31 03:49:52.575527 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.575506    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.656332 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.656303    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"]
Oct 31 03:49:52.675111 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.675083    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-ae8e27\"/\"distinct-w0-ae8e27-kserve-self-signed-certs\""
Oct 31 03:49:52.714373 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.714340    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-model-cache\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.714532 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.714382    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-dshm\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.714532 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.714501    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-tmp-dir\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.714624 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.714610    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f0348bd-b29e-49eb-802a-dfcb95413b8a-tls-certs\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.714679 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.714661    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-home\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.714772 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.714754    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2bkd\" (UniqueName: \"kubernetes.io/projected/9f0348bd-b29e-49eb-802a-dfcb95413b8a-kube-api-access-w2bkd\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.815938 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.815900    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f0348bd-b29e-49eb-802a-dfcb95413b8a-tls-certs\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.816152 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.815973    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-home\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.816152 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.816016    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2bkd\" (UniqueName: \"kubernetes.io/projected/9f0348bd-b29e-49eb-802a-dfcb95413b8a-kube-api-access-w2bkd\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.816152 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.816052    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-model-cache\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.816152 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.816082    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-dshm\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.816152 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.816125    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-tmp-dir\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.816566 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.816542    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-home\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.816656 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.816576    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-model-cache\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.816656 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.816634    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-tmp-dir\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.818580 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.818555    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-dshm\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.818694 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.818648    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f0348bd-b29e-49eb-802a-dfcb95413b8a-tls-certs\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.875481 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.875405    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2bkd\" (UniqueName: \"kubernetes.io/projected/9f0348bd-b29e-49eb-802a-dfcb95413b8a-kube-api-access-w2bkd\") pod \"distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") " pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:52.888178 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:52.888147    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:49:53.287381 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:53.287359    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"]
Oct 31 03:49:53.288749 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:49:53.288721    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f0348bd_b29e_49eb_802a_dfcb95413b8a.slice/crio-fa879cd910817c5b380a1cdb9aa0618de4404e3cd4383f534fdfc28e39924580 WatchSource:0}: Error finding container fa879cd910817c5b380a1cdb9aa0618de4404e3cd4383f534fdfc28e39924580: Status 404 returned error can't find the container with id fa879cd910817c5b380a1cdb9aa0618de4404e3cd4383f534fdfc28e39924580
Oct 31 03:49:53.581503 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:53.581418    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:53.594097 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:53.594073    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:54.123624 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:54.123587    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" event={"ID":"9f0348bd-b29e-49eb-802a-dfcb95413b8a","Type":"ContainerStarted","Data":"d7bae8971d3f73a64c66de43ed6039e25719a477b0bd614f7f932d706b8dd097"}
Oct 31 03:49:54.123802 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:54.123632    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" event={"ID":"9f0348bd-b29e-49eb-802a-dfcb95413b8a","Type":"ContainerStarted","Data":"fa879cd910817c5b380a1cdb9aa0618de4404e3cd4383f534fdfc28e39924580"}
Oct 31 03:49:54.134995 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:54.134969    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:49:54.236355 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:54.236294    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" podStartSLOduration=2.236276493 podStartE2EDuration="2.236276493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:49:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:49:54.234243461 +0000 UTC m=+1290.819783725" watchObservedRunningTime="2026-10-31 03:49:54.236276493 +0000 UTC m=+1290.821816748"
Oct 31 03:49:54.888289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:49:54.888230    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:50:02.888466 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:02.888382    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:50:02.901557 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:02.901531    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:50:03.180941 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:03.180861    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:50:05.313351 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:50:05.313311    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:50:35.314496 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:50:35.314453    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:50:37.673226 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.673188    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"]
Oct 31 03:50:37.678795 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.678777    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.709467 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.709435    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.709628 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.709501    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.709628 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.709608    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.709716 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.709694    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.709845 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.709826    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.709919 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.709887    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.710006 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.709976    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.710067 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.710049    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w4r2\" (UniqueName: \"kubernetes.io/projected/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-kube-api-access-7w4r2\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.710125 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.710111    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.811222 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.811187    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.811222 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.811237    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.811453 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.811261    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.811453 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.811347    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.811453 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.811430    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.811585 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.811483    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.811585 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.811508    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.811585 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.811538    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w4r2\" (UniqueName: \"kubernetes.io/projected/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-kube-api-access-7w4r2\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.811740 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.811588    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.811740 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.811675    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-credential-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.811848 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.811778    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-data\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.811905 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.811876    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-workload-certs\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.811905 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.811892    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-workload-socket\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.812414 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.812254    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istiod-ca-cert\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.813873 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.813849    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-podinfo\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.814648 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.814629    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-envoy\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.870303 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.870251    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-token\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.870303 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.870288    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w4r2\" (UniqueName: \"kubernetes.io/projected/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-kube-api-access-7w4r2\") pod \"e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") " pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:37.877295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.877233    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"]
Oct 31 03:50:37.992340 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:37.992302    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:38.416219 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:38.416191    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"]
Oct 31 03:50:38.417917 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:50:38.417887    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69775e74_fc6d_4f0d_9f2e_b8e291ca00b0.slice/crio-daf7c6f50db010945ad5f3d0246b6cd48862d53e65fec4713a7d2ab669376631 WatchSource:0}: Error finding container daf7c6f50db010945ad5f3d0246b6cd48862d53e65fec4713a7d2ab669376631: Status 404 returned error can't find the container with id daf7c6f50db010945ad5f3d0246b6cd48862d53e65fec4713a7d2ab669376631
Oct 31 03:50:38.420466 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:38.420435    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:50:38.420555 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:38.420500    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:50:38.420555 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:38.420528    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:50:39.356530 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:39.356483    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448" event={"ID":"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0","Type":"ContainerStarted","Data":"cfc6fb2817a10e6fbdb17ef303e8976b303dbbf43af4c322e6f49086f5a8fdbd"}
Oct 31 03:50:39.357023 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:39.356558    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448" event={"ID":"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0","Type":"ContainerStarted","Data":"daf7c6f50db010945ad5f3d0246b6cd48862d53e65fec4713a7d2ab669376631"}
Oct 31 03:50:39.524241 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:39.524171    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448" podStartSLOduration=2.5241506190000003 podStartE2EDuration="2.524150619s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:50:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:50:39.522806484 +0000 UTC m=+1336.108346770" watchObservedRunningTime="2026-10-31 03:50:39.524150619 +0000 UTC m=+1336.109690886"
Oct 31 03:50:39.993509 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:39.993472    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:39.994835 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:39.994810    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 31 03:50:39.994950 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:39.994860    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448" podUID="69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 31 03:50:40.993057 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:40.993026    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 31 03:50:40.993476 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:40.993084    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448" podUID="69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 31 03:50:41.993412 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:41.993379    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 31 03:50:41.993805 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:41.993433    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448" podUID="69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 31 03:50:42.993669 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:42.993631    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 31 03:50:42.994094 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:42.993688    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448" podUID="69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 31 03:50:43.993046 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:43.993007    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 31 03:50:43.993234 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:43.993081    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448" podUID="69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 31 03:50:44.993437 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:44.993399    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Oct 31 03:50:44.993828 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:44.993465    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448" podUID="69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Oct 31 03:50:45.997750 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:45.997719    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:45.998173 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:45.998011    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:45.998683 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:45.998652    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 03:50:56.910622 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:56.910590    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"]
Oct 31 03:50:56.914523 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:56.914500    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.011791 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.011760    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-e5b8f9\"/\"distinct-2-w5-e5b8f9-kserve-self-signed-certs\""
Oct 31 03:50:57.014176 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.014152    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"]
Oct 31 03:50:57.015429 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.015410    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-tmp-dir\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.015486 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.015455    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-dshm\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.015486 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.015476    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7rng\" (UniqueName: \"kubernetes.io/projected/067339e5-0df5-421d-8b14-471a9b1a9735-kube-api-access-k7rng\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.015619 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.015600    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-home\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.015660 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.015647    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-model-cache\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.015767 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.015750    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/067339e5-0df5-421d-8b14-471a9b1a9735-tls-certs\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.117113 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.117082    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/067339e5-0df5-421d-8b14-471a9b1a9735-tls-certs\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.117316 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.117153    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-tmp-dir\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.117316 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.117191    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-dshm\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.117316 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.117213    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7rng\" (UniqueName: \"kubernetes.io/projected/067339e5-0df5-421d-8b14-471a9b1a9735-kube-api-access-k7rng\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.117472 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.117317    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-home\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.117668 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.117643    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-tmp-dir\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.117861 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.117683    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-home\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.117861 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.117724    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-model-cache\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.118208 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.117946    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-model-cache\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.119691 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.119669    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-dshm\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.119826 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.119807    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/067339e5-0df5-421d-8b14-471a9b1a9735-tls-certs\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.176112 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.176021    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7rng\" (UniqueName: \"kubernetes.io/projected/067339e5-0df5-421d-8b14-471a9b1a9735-kube-api-access-k7rng\") pod \"distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") " pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.226571 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.226535    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:50:57.612382 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:57.612344    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"]
Oct 31 03:50:57.613589 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:50:57.613566    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod067339e5_0df5_421d_8b14_471a9b1a9735.slice/crio-e1616542656096d18c48186cfa6b271ccc74e3b8c59ad1990e94c2b14ae5f644 WatchSource:0}: Error finding container e1616542656096d18c48186cfa6b271ccc74e3b8c59ad1990e94c2b14ae5f644: Status 404 returned error can't find the container with id e1616542656096d18c48186cfa6b271ccc74e3b8c59ad1990e94c2b14ae5f644
Oct 31 03:50:58.457407 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:58.457371    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg" event={"ID":"067339e5-0df5-421d-8b14-471a9b1a9735","Type":"ContainerStarted","Data":"d5bd5a2191e0ed712c33d2bb0a24980e20245d6706163be9f493a31a406975d3"}
Oct 31 03:50:58.458111 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:58.457417    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg" event={"ID":"067339e5-0df5-421d-8b14-471a9b1a9735","Type":"ContainerStarted","Data":"e1616542656096d18c48186cfa6b271ccc74e3b8c59ad1990e94c2b14ae5f644"}
Oct 31 03:50:58.569443 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:58.569385    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg" podStartSLOduration=2.569370708 podStartE2EDuration="2.569370708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:50:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:50:58.568166105 +0000 UTC m=+1355.153706366" watchObservedRunningTime="2026-10-31 03:50:58.569370708 +0000 UTC m=+1355.154910970"
Oct 31 03:50:59.227155 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:50:59.227124    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:51:05.312622 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:51:05.312582    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:51:07.227150 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:51:07.227115    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:51:07.246180 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:51:07.246150    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:51:07.518632 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:51:07.518553    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:51:35.314322 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:51:35.314280    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:51:59.977655 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:51:59.977618    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"]
Oct 31 03:51:59.986213 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:51:59.986185    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.039132 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.039097    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"]
Oct 31 03:52:00.087408 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.087378    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-537e34\"/\"distinct-2-w3-537e34-kserve-self-signed-certs\""
Oct 31 03:52:00.099991 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.099960    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b319682-afe9-4edd-b9c0-46d7cb7a989c-tls-certs\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.100153 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.100020    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-home\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.100153 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.100110    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvk8r\" (UniqueName: \"kubernetes.io/projected/4b319682-afe9-4edd-b9c0-46d7cb7a989c-kube-api-access-qvk8r\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.100153 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.100138    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-tmp-dir\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.100357 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.100232    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-dshm\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.100357 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.100336    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-model-cache\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.201295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.201246    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-dshm\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.201479 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.201337    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-model-cache\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.201479 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.201439    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b319682-afe9-4edd-b9c0-46d7cb7a989c-tls-certs\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.201597 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.201477    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-home\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.201597 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.201504    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvk8r\" (UniqueName: \"kubernetes.io/projected/4b319682-afe9-4edd-b9c0-46d7cb7a989c-kube-api-access-qvk8r\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.201705 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.201609    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-tmp-dir\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.201877 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.201848    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-home\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.202021 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.201934    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-tmp-dir\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.202084 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.202029    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-model-cache\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.203858 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.203835    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-dshm\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.204063 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.204045    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b319682-afe9-4edd-b9c0-46d7cb7a989c-tls-certs\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.272451 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.272363    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvk8r\" (UniqueName: \"kubernetes.io/projected/4b319682-afe9-4edd-b9c0-46d7cb7a989c-kube-api-access-qvk8r\") pod \"distinct-2-w3-537e34-kserve-68448d4986-95r59\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") " pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.298474 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.298442    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:00.669239 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:52:00.669206    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b319682_afe9_4edd_b9c0_46d7cb7a989c.slice/crio-698f230b627d5e11895ce9f9111b04d60a0aa4d3a2819bb142b842b7c687ebfb WatchSource:0}: Error finding container 698f230b627d5e11895ce9f9111b04d60a0aa4d3a2819bb142b842b7c687ebfb: Status 404 returned error can't find the container with id 698f230b627d5e11895ce9f9111b04d60a0aa4d3a2819bb142b842b7c687ebfb
Oct 31 03:52:00.673345 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.673320    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"]
Oct 31 03:52:00.789051 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.789016    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59" event={"ID":"4b319682-afe9-4edd-b9c0-46d7cb7a989c","Type":"ContainerStarted","Data":"637be0d48342e518ce989be5370f34f1061ce0b2b0bdddb3252baa127b5c5022"}
Oct 31 03:52:00.789215 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.789058    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59" event={"ID":"4b319682-afe9-4edd-b9c0-46d7cb7a989c","Type":"ContainerStarted","Data":"698f230b627d5e11895ce9f9111b04d60a0aa4d3a2819bb142b842b7c687ebfb"}
Oct 31 03:52:00.912689 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:00.912628    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59" podStartSLOduration=1.912610184 podStartE2EDuration="1.912610184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:51:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:52:00.911196192 +0000 UTC m=+1417.496736454" watchObservedRunningTime="2026-10-31 03:52:00.912610184 +0000 UTC m=+1417.498150447"
Oct 31 03:52:01.298825 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:01.298794    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:05.316116 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:52:05.316082    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:52:10.298985 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:10.298941    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:10.313020 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:10.312991    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:10.889581 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:10.889554    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:52:23.017110 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.017074    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"]
Oct 31 03:52:23.020708 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.020691    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.075178 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.075151    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-68aa3e-openshift-default-dockercfg-jktfb\""
Oct 31 03:52:23.077689 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.077666    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.077764 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.077702    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.077814 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.077787    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.077859 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.077837    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.077904 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.077868    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.078017 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.078001    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.078088 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.078067    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkcmz\" (UniqueName: \"kubernetes.io/projected/9bdad2ce-6f22-4a30-b7e2-328bededd05b-kube-api-access-kkcmz\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.078127 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.078101    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.078178 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.078126    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.129433 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.129396    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"]
Oct 31 03:52:23.178796 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.178762    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.179032 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.178815    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkcmz\" (UniqueName: \"kubernetes.io/projected/9bdad2ce-6f22-4a30-b7e2-328bededd05b-kube-api-access-kkcmz\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.179032 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.178837    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.179032 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.178859    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.179032 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.179000    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.179322 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.179052    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.179322 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.179116    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.179322 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.179143    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.179322 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.179179    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.179322 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.179198    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.179570 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.179417    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.179570 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.179513    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.179647 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.179574    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.179838 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.179817    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.181517 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.181498    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.181898 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.181879    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.241883 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.241858    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.242008 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.241936    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkcmz\" (UniqueName: \"kubernetes.io/projected/9bdad2ce-6f22-4a30-b7e2-328bededd05b-kube-api-access-kkcmz\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.335541 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.335447    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:23.756042 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:52:23.756007    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bdad2ce_6f22_4a30_b7e2_328bededd05b.slice/crio-753ba049068f2eaab7aac077adc031957bdbbe2282571a4850596eebd3804c20 WatchSource:0}: Error finding container 753ba049068f2eaab7aac077adc031957bdbbe2282571a4850596eebd3804c20: Status 404 returned error can't find the container with id 753ba049068f2eaab7aac077adc031957bdbbe2282571a4850596eebd3804c20
Oct 31 03:52:23.758559 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.758526    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:52:23.758645 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.758592    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:52:23.758645 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.758619    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:52:23.759204 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.759185    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"]
Oct 31 03:52:23.966950 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.966909    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp" event={"ID":"9bdad2ce-6f22-4a30-b7e2-328bededd05b","Type":"ContainerStarted","Data":"3da33df7600481fecb67ea416981acd962ee48a92e6d881b8bff17e12d24aeae"}
Oct 31 03:52:23.967176 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:23.966974    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp" event={"ID":"9bdad2ce-6f22-4a30-b7e2-328bededd05b","Type":"ContainerStarted","Data":"753ba049068f2eaab7aac077adc031957bdbbe2282571a4850596eebd3804c20"}
Oct 31 03:52:24.093884 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:24.093609    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp" podStartSLOduration=2.093589842 podStartE2EDuration="2.093589842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:52:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:52:24.093420742 +0000 UTC m=+1440.678961006" watchObservedRunningTime="2026-10-31 03:52:24.093589842 +0000 UTC m=+1440.679130106"
Oct 31 03:52:24.336502 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:24.336468    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:24.338381 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:24.338357    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 31 03:52:24.338466 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:24.338410    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp" podUID="9bdad2ce-6f22-4a30-b7e2-328bededd05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 31 03:52:25.336030 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:25.335995    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 31 03:52:25.336534 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:25.336048    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp" podUID="9bdad2ce-6f22-4a30-b7e2-328bededd05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 31 03:52:26.336457 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:26.336425    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 31 03:52:26.336881 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:26.336478    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp" podUID="9bdad2ce-6f22-4a30-b7e2-328bededd05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 31 03:52:27.336848 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:27.336810    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 31 03:52:27.337355 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:27.336871    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp" podUID="9bdad2ce-6f22-4a30-b7e2-328bededd05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 31 03:52:28.336887 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:28.336845    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 31 03:52:28.337323 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:28.336916    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp" podUID="9bdad2ce-6f22-4a30-b7e2-328bededd05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 31 03:52:29.336353 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:29.336311    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 31 03:52:29.336533 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:29.336389    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp" podUID="9bdad2ce-6f22-4a30-b7e2-328bededd05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 31 03:52:30.338729 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:30.338690    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Oct 31 03:52:30.339397 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:30.339362    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp" podUID="9bdad2ce-6f22-4a30-b7e2-328bededd05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Oct 31 03:52:31.339842 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:31.339815    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:31.340389 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:31.340101    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:31.340962 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:31.340944    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 03:52:35.315280 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:52:35.315226    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:52:52.618998 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.618958    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"]
Oct 31 03:52:52.623432 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.623411    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.723604 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.723573    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.723604 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.723606    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.723870 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.723628    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.723870 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.723773    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77c6a418-701d-4061-b6db-37cf20c502e4-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.723870 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.723839    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c6a418-701d-4061-b6db-37cf20c502e4-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.724043 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.723870    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxm98\" (UniqueName: \"kubernetes.io/projected/77c6a418-701d-4061-b6db-37cf20c502e4-kube-api-access-sxm98\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.724043 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.723985    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.724043 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.724018    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.724197 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.724076    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77c6a418-701d-4061-b6db-37cf20c502e4-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.729365 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.729339    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-ae8e27-openshift-default-dockercfg-sgjl6\""
Oct 31 03:52:52.731982 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.731957    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"]
Oct 31 03:52:52.824964 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.824928    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.825134 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.824970    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.825134 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.825029    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77c6a418-701d-4061-b6db-37cf20c502e4-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.825299 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.825147    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.825299 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.825197    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.825299 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.825227    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.825463 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.825323    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77c6a418-701d-4061-b6db-37cf20c502e4-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.825463 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.825363    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.825463 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.825367    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c6a418-701d-4061-b6db-37cf20c502e4-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.825463 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.825412    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.825463 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.825434    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxm98\" (UniqueName: \"kubernetes.io/projected/77c6a418-701d-4061-b6db-37cf20c502e4-kube-api-access-sxm98\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.825729 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.825518    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.825729 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.825601    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.825836 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.825789    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77c6a418-701d-4061-b6db-37cf20c502e4-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.827768 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.827743    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.828045 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.828027    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77c6a418-701d-4061-b6db-37cf20c502e4-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.888115 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.888054    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c6a418-701d-4061-b6db-37cf20c502e4-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.888229 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.888112    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxm98\" (UniqueName: \"kubernetes.io/projected/77c6a418-701d-4061-b6db-37cf20c502e4-kube-api-access-sxm98\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:52.939923 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:52.939897    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:53.030791 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.030760    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"]
Oct 31 03:52:53.035482 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.035451    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.091590 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.091561    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"]
Oct 31 03:52:53.128668 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.128640    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.128823 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.128689    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.128823 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.128710    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.128925 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.128834    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.128925 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.128881    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.129037 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.129021    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.129075 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.129048    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.129129 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.129093    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj5fb\" (UniqueName: \"kubernetes.io/projected/a977c9a1-fe79-4f13-8aa0-9a3484a28238-kube-api-access-tj5fb\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.129213 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.129195    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.230257 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.230228    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.230257 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.230263    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.230492 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.230341    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj5fb\" (UniqueName: \"kubernetes.io/projected/a977c9a1-fe79-4f13-8aa0-9a3484a28238-kube-api-access-tj5fb\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.230492 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.230385    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.230492 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.230437    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.230629 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.230509    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.230629 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.230537    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.230751 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.230671    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.230751 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.230727    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.230875 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.230858    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.230933 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.230913    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.231050 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.231014    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.231176 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.231078    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.231301 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.231253    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.232948 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.232927    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.233196 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.233180    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.293351 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.293324    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.293556 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.293535    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj5fb\" (UniqueName: \"kubernetes.io/projected/a977c9a1-fe79-4f13-8aa0-9a3484a28238-kube-api-access-tj5fb\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.349918 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.349889    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:53.360794 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:52:53.360767    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77c6a418_701d_4061_b6db_37cf20c502e4.slice/crio-d56a45a755337a16a8d70a18104b99f47af3fe37fdf526ed9bfc2db88c4b334e WatchSource:0}: Error finding container d56a45a755337a16a8d70a18104b99f47af3fe37fdf526ed9bfc2db88c4b334e: Status 404 returned error can't find the container with id d56a45a755337a16a8d70a18104b99f47af3fe37fdf526ed9bfc2db88c4b334e
Oct 31 03:52:53.362848 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.362752    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"]
Oct 31 03:52:53.363756 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.363728    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:52:53.363838 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.363786    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:52:53.363838 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.363816    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:52:53.781222 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:52:53.781186    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda977c9a1_fe79_4f13_8aa0_9a3484a28238.slice/crio-22e043e78ac64829e73c6de8a043b081790d719d942160d2d79acb28799dfb89 WatchSource:0}: Error finding container 22e043e78ac64829e73c6de8a043b081790d719d942160d2d79acb28799dfb89: Status 404 returned error can't find the container with id 22e043e78ac64829e73c6de8a043b081790d719d942160d2d79acb28799dfb89
Oct 31 03:52:53.783961 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.783928    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:52:53.784075 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.784010    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:52:53.784075 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.784052    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:52:53.784642 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:53.784615    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"]
Oct 31 03:52:54.202687 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:54.202655    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5" event={"ID":"a977c9a1-fe79-4f13-8aa0-9a3484a28238","Type":"ContainerStarted","Data":"4f46e0ba4ab20e1fe4cc3f07ee82c9f9c858fb5edbc39ccb553b5562d29245b5"}
Oct 31 03:52:54.202884 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:54.202718    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5" event={"ID":"a977c9a1-fe79-4f13-8aa0-9a3484a28238","Type":"ContainerStarted","Data":"22e043e78ac64829e73c6de8a043b081790d719d942160d2d79acb28799dfb89"}
Oct 31 03:52:54.204400 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:54.204376    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6" event={"ID":"77c6a418-701d-4061-b6db-37cf20c502e4","Type":"ContainerStarted","Data":"ebcb63fefcafe68d0df8d85efc70cab0c2e03e1546130e1066f4966d3da1e594"}
Oct 31 03:52:54.204508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:54.204416    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6" event={"ID":"77c6a418-701d-4061-b6db-37cf20c502e4","Type":"ContainerStarted","Data":"d56a45a755337a16a8d70a18104b99f47af3fe37fdf526ed9bfc2db88c4b334e"}
Oct 31 03:52:54.350940 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:54.350903    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:52:54.352678 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:54.352653    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 31 03:52:54.352816 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:54.352701    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5" podUID="a977c9a1-fe79-4f13-8aa0-9a3484a28238" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 31 03:52:54.466017 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:54.465896    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5" podStartSLOduration=2.465879717 podStartE2EDuration="2.465879717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:52:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:52:54.336385993 +0000 UTC m=+1470.921926271" watchObservedRunningTime="2026-10-31 03:52:54.465879717 +0000 UTC m=+1471.051419980"
Oct 31 03:52:54.467990 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:54.467933    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6" podStartSLOduration=2.467918577 podStartE2EDuration="2.467918577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:52:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:52:54.464787874 +0000 UTC m=+1471.050328148" watchObservedRunningTime="2026-10-31 03:52:54.467918577 +0000 UTC m=+1471.053458839"
Oct 31 03:52:54.942351 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:54.941640    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:52:54.958295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:54.957920    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 31 03:52:54.958295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:54.958018    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6" podUID="77c6a418-701d-4061-b6db-37cf20c502e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 31 03:52:55.357292 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:55.353247    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 31 03:52:55.357292 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:55.353359    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5" podUID="a977c9a1-fe79-4f13-8aa0-9a3484a28238" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 31 03:52:55.949294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:55.944756    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 31 03:52:55.949294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:55.944829    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6" podUID="77c6a418-701d-4061-b6db-37cf20c502e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 31 03:52:56.358349 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:56.352657    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 31 03:52:56.358349 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:56.352719    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5" podUID="a977c9a1-fe79-4f13-8aa0-9a3484a28238" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 31 03:52:56.953297 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:56.952959    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 31 03:52:56.953297 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:56.953032    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6" podUID="77c6a418-701d-4061-b6db-37cf20c502e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 31 03:52:57.350767 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:57.350672    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 31 03:52:57.351045 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:57.351016    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5" podUID="a977c9a1-fe79-4f13-8aa0-9a3484a28238" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 31 03:52:57.940434 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:57.940397    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 31 03:52:57.940687 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:57.940656    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6" podUID="77c6a418-701d-4061-b6db-37cf20c502e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 31 03:52:58.352758 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:58.352708    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 31 03:52:58.353305 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:58.352781    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5" podUID="a977c9a1-fe79-4f13-8aa0-9a3484a28238" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 31 03:52:58.958319 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:58.951385    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 31 03:52:58.958319 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:58.951450    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6" podUID="77c6a418-701d-4061-b6db-37cf20c502e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 31 03:52:59.350743 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:59.350639    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 31 03:52:59.350998 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:59.350968    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5" podUID="a977c9a1-fe79-4f13-8aa0-9a3484a28238" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 31 03:52:59.940766 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:59.940731    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 31 03:52:59.941243 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:52:59.940789    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6" podUID="77c6a418-701d-4061-b6db-37cf20c502e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 31 03:53:00.351236 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:00.351195    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 31 03:53:00.351431 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:00.351297    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5" podUID="a977c9a1-fe79-4f13-8aa0-9a3484a28238" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 31 03:53:00.946865 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:00.946834    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:53:00.947357 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:00.947318    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:53:00.948242 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:00.948218    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:53:01.355029 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:01.354998    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:53:01.355536 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:01.355473    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:53:01.356429 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:01.356409    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 03:53:05.316308 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:53:05.316205    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:53:07.745562 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.745526    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"]
Oct 31 03:53:07.750688 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.750668    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.843116 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.843082    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.843298 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.843132    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.843349 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.843312    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.843389 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.843358    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71787759-a0b2-43c1-9939-acfc039ca1eb-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.843476 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.843460    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.843510 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.843488    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts2n4\" (UniqueName: \"kubernetes.io/projected/71787759-a0b2-43c1-9939-acfc039ca1eb-kube-api-access-ts2n4\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.843571 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.843559    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.843604 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.843583    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.843650 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.843603    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.859019 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.858988    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"]
Oct 31 03:53:07.944563 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.944522    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.944742 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.944573    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71787759-a0b2-43c1-9939-acfc039ca1eb-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.944784 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.944751    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.944836 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.944788    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts2n4\" (UniqueName: \"kubernetes.io/projected/71787759-a0b2-43c1-9939-acfc039ca1eb-kube-api-access-ts2n4\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.944836 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.944828    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.944920 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.944859    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.944920 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.944892    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.945025 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.944988    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.945092 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.945039    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.945322 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.945232    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.945453 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.945343    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.945531 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.945448    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.945531 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.945490    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.945610 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.945571    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71787759-a0b2-43c1-9939-acfc039ca1eb-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.947206 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.947180    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:07.947326 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:07.947219    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:08.007888 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.007823    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:08.008029 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.007878    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts2n4\" (UniqueName: \"kubernetes.io/projected/71787759-a0b2-43c1-9939-acfc039ca1eb-kube-api-access-ts2n4\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:08.063929 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.063898    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:08.132386 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.132351    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"]
Oct 31 03:53:08.141685 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.141339    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.189007 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.188974    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"]
Oct 31 03:53:08.248857 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.248824    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.249036 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.248934    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.249036 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.249005    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.249181 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.249163    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.249244 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.249203    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95svf\" (UniqueName: \"kubernetes.io/projected/a9f9a022-fa1c-4f08-ac54-cb751f629547-kube-api-access-95svf\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.249244 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.249234    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.249376 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.249299    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.249414 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.249378    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.249414 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.249398    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9f9a022-fa1c-4f08-ac54-cb751f629547-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.351103 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.351013    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.351103 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.351058    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9f9a022-fa1c-4f08-ac54-cb751f629547-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.351103 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.351086    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.351427 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.351122    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.351427 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.351176    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.351427 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.351281    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.351427 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.351315    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95svf\" (UniqueName: \"kubernetes.io/projected/a9f9a022-fa1c-4f08-ac54-cb751f629547-kube-api-access-95svf\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.351427 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.351343    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.351427 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.351375    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.351784 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.351760    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-credential-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.351923 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.351865    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-workload-certs\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.351992 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.351937    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-workload-socket\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.351992 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.351884    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-data\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.352152 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.352133    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9f9a022-fa1c-4f08-ac54-cb751f629547-istiod-ca-cert\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.353563 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.353540    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-envoy\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.353938 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.353917    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-podinfo\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.414436 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.414398    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-token\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.414570 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.414521    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95svf\" (UniqueName: \"kubernetes.io/projected/a9f9a022-fa1c-4f08-ac54-cb751f629547-kube-api-access-95svf\") pod \"e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") " pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.457927 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.457898    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:08.482103 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:53:08.482070    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71787759_a0b2_43c1_9939_acfc039ca1eb.slice/crio-284266cfeca6a629344a6f4e9b66a717bcc04d0c93ac8a7b0fded40b6abccca9 WatchSource:0}: Error finding container 284266cfeca6a629344a6f4e9b66a717bcc04d0c93ac8a7b0fded40b6abccca9: Status 404 returned error can't find the container with id 284266cfeca6a629344a6f4e9b66a717bcc04d0c93ac8a7b0fded40b6abccca9
Oct 31 03:53:08.484399 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.484360    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:53:08.484496 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.484435    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:53:08.484496 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.484474    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:53:08.485801 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.485778    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"]
Oct 31 03:53:08.836969 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:53:08.836935    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9f9a022_fa1c_4f08_ac54_cb751f629547.slice/crio-84e80ad0822c95ad77aa867e8e4f90cf443d6c180cca7a7933bca9570657db78 WatchSource:0}: Error finding container 84e80ad0822c95ad77aa867e8e4f90cf443d6c180cca7a7933bca9570657db78: Status 404 returned error can't find the container with id 84e80ad0822c95ad77aa867e8e4f90cf443d6c180cca7a7933bca9570657db78
Oct 31 03:53:08.839700 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.839667    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:53:08.839794 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.839735    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:53:08.839794 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.839769    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:53:08.841713 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:08.841691    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"]
Oct 31 03:53:09.346131 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.346090    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc" event={"ID":"71787759-a0b2-43c1-9939-acfc039ca1eb","Type":"ContainerStarted","Data":"86fe3d311bfdd7fc2ccdd3b61d5e630e4646bb4998a9e8e720e2f04b1151679e"}
Oct 31 03:53:09.346390 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.346186    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc" event={"ID":"71787759-a0b2-43c1-9939-acfc039ca1eb","Type":"ContainerStarted","Data":"284266cfeca6a629344a6f4e9b66a717bcc04d0c93ac8a7b0fded40b6abccca9"}
Oct 31 03:53:09.348176 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.348146    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" event={"ID":"a9f9a022-fa1c-4f08-ac54-cb751f629547","Type":"ContainerStarted","Data":"1c5b41c24dc4419305a9902fd7d8c402b3c7ddc9f4b0594ed678e4338247fb17"}
Oct 31 03:53:09.348343 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.348189    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" event={"ID":"a9f9a022-fa1c-4f08-ac54-cb751f629547","Type":"ContainerStarted","Data":"84e80ad0822c95ad77aa867e8e4f90cf443d6c180cca7a7933bca9570657db78"}
Oct 31 03:53:09.354411 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.354344    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"]
Oct 31 03:53:09.358653 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.358632    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.449491 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.449457    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"]
Oct 31 03:53:09.458864 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.458821    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:09.460367 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.460322    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 31 03:53:09.460501 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.460393    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" podUID="a9f9a022-fa1c-4f08-ac54-cb751f629547" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 31 03:53:09.467706 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.467678    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.467824 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.467803    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a65ad9de-668e-4600-8643-546a6ce94b99-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.467864 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.467847    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxmvl\" (UniqueName: \"kubernetes.io/projected/a65ad9de-668e-4600-8643-546a6ce94b99-kube-api-access-mxmvl\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.467898 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.467891    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.468071 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.468052    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.468231 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.468211    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.468326 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.468246    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a65ad9de-668e-4600-8643-546a6ce94b99-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.468389 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.468329    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a65ad9de-668e-4600-8643-546a6ce94b99-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.468604 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.468464    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.474231 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.474187    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc" podStartSLOduration=2.474174709 podStartE2EDuration="2.474174709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:53:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:53:09.470726655 +0000 UTC m=+1486.056266919" watchObservedRunningTime="2026-10-31 03:53:09.474174709 +0000 UTC m=+1486.059714971"
Oct 31 03:53:09.569536 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.569499    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.569724 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.569605    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.569724 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.569647    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a65ad9de-668e-4600-8643-546a6ce94b99-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.569724 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.569673    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxmvl\" (UniqueName: \"kubernetes.io/projected/a65ad9de-668e-4600-8643-546a6ce94b99-kube-api-access-mxmvl\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.569724 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.569704    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.569943 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.569799    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.569943 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.569839    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.569943 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.569862    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a65ad9de-668e-4600-8643-546a6ce94b99-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.569943 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.569901    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a65ad9de-668e-4600-8643-546a6ce94b99-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.570148 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.569942    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.570148 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.570087    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.570227 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.570155    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.570402 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.570372    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.570765 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.570745    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a65ad9de-668e-4600-8643-546a6ce94b99-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.572482 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.572459    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.572590 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.572573    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a65ad9de-668e-4600-8643-546a6ce94b99-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.598583 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.598491    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" podStartSLOduration=1.5984776250000001 podStartE2EDuration="1.598477625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:53:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:53:09.595933148 +0000 UTC m=+1486.181473413" watchObservedRunningTime="2026-10-31 03:53:09.598477625 +0000 UTC m=+1486.184017886"
Oct 31 03:53:09.641894 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.641857    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a65ad9de-668e-4600-8643-546a6ce94b99-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.642113 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.642092    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxmvl\" (UniqueName: \"kubernetes.io/projected/a65ad9de-668e-4600-8643-546a6ce94b99-kube-api-access-mxmvl\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:09.674200 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:09.674149    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:10.065378 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:10.065077    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:10.069805 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:10.069451    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 31 03:53:10.069805 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:10.069510    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc" podUID="71787759-a0b2-43c1-9939-acfc039ca1eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 31 03:53:10.168671 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:10.165964    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"]
Oct 31 03:53:10.178515 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:10.174438    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:53:10.178515 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:10.174550    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:53:10.178515 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:10.174615    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:53:10.373258 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:10.373123    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz" event={"ID":"a65ad9de-668e-4600-8643-546a6ce94b99","Type":"ContainerStarted","Data":"b31caaadab9c9cb7d86cd705e93bdead0a8a4ec3ccf7d9b4a71c806d0c85837d"}
Oct 31 03:53:10.373258 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:10.373205    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz" event={"ID":"a65ad9de-668e-4600-8643-546a6ce94b99","Type":"ContainerStarted","Data":"3a430a2ccba9d6d78b03d9eeb062abee83364abef27360694ecdf3949c8e8a79"}
Oct 31 03:53:10.462294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:10.461671    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 31 03:53:10.462294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:10.461733    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" podUID="a9f9a022-fa1c-4f08-ac54-cb751f629547" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 31 03:53:10.503016 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:10.502031    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz" podStartSLOduration=1.502011996 podStartE2EDuration="1.502011996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:53:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:53:10.500077905 +0000 UTC m=+1487.085618167" watchObservedRunningTime="2026-10-31 03:53:10.502011996 +0000 UTC m=+1487.087552263"
Oct 31 03:53:10.675397 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:10.675282    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:11.065293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:11.064636    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 31 03:53:11.065293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:11.064699    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc" podUID="71787759-a0b2-43c1-9939-acfc039ca1eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 31 03:53:11.459737 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:11.459702    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 31 03:53:11.460008 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:11.459982    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" podUID="a9f9a022-fa1c-4f08-ac54-cb751f629547" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 31 03:53:11.679294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:11.678153    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 31 03:53:11.679294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:11.678215    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz" podUID="a65ad9de-668e-4600-8643-546a6ce94b99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 31 03:53:12.102302 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:12.101991    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 31 03:53:12.102302 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:12.102037    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc" podUID="71787759-a0b2-43c1-9939-acfc039ca1eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 31 03:53:12.507294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:12.483604    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 31 03:53:12.507294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:12.483676    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" podUID="a9f9a022-fa1c-4f08-ac54-cb751f629547" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 31 03:53:12.678303 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:12.674920    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 31 03:53:12.679031 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:12.678993    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz" podUID="a65ad9de-668e-4600-8643-546a6ce94b99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 31 03:53:13.068292 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.067751    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 31 03:53:13.068292 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.067816    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc" podUID="71787759-a0b2-43c1-9939-acfc039ca1eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 31 03:53:13.145293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.144738    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:13.207293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.195867    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Oct 31 03:53:13.237291 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.233395    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4" podUID="6fc591ce-8543-4590-9dd5-9002226418d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:13.474288 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.473403    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj" podUID="a5ad5303-15e7-422a-a144-5598d90f33db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.41:8000/ready\": context deadline exceeded"
Oct 31 03:53:13.474766 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.474738    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 31 03:53:13.474946 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.474923    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" podUID="a9f9a022-fa1c-4f08-ac54-cb751f629547" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 31 03:53:13.528408 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.528361    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg" podUID="067339e5-0df5-421d-8b14-471a9b1a9735" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:13.546415 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.546372    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" podUID="564888f8-35bb-4028-8040-110c959c9b19" containerName="main" probeResult="failure" output="Get \"https://10.132.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:13.614410 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.614357    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" podUID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:13.631414 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.631361    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:13.632069 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.632037    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l" podUID="a5f198d8-39a2-4c72-87de-516e077d47bd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Oct 31 03:53:13.675398 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.675354    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" podUID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Oct 31 03:53:13.676838 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.676809    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 31 03:53:13.677038 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.677012    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz" podUID="a65ad9de-668e-4600-8643-546a6ce94b99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 31 03:53:13.888459 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.888358    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59" podUID="4b319682-afe9-4edd-b9c0-46d7cb7a989c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:13.889116 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.888704    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:13.956294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.955350    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j" podUID="f97017f6-52c4-4d27-9aa7-5045b9ac85ee" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:13.973294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:13.970382    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" podUID="ade2ace8-0217-404c-9592-85576f16b085" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.068109 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.068075    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 31 03:53:14.069410 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.069376    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc" podUID="71787759-a0b2-43c1-9939-acfc039ca1eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 31 03:53:14.146349 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.140353    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn" podUID="e20732a1-da5b-4e22-b861-6b03dfe62522" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.146962 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.146922    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.147541 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.147519    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:53:14.203338 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.203299    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.203977 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.203952    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:53:14.242381 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.242341    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4" podUID="6fc591ce-8543-4590-9dd5-9002226418d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.244398 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.243688    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:53:14.310138 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.309361    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j" podUID="f97017f6-52c4-4d27-9aa7-5045b9ac85ee" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.380302 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.364008    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn" podUID="e20732a1-da5b-4e22-b861-6b03dfe62522" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.667293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.503645    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj" podUID="a5ad5303-15e7-422a-a144-5598d90f33db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.667293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.533423    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg" podUID="067339e5-0df5-421d-8b14-471a9b1a9735" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.667293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.658226    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:53:14.667293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.658314    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:53:14.667293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.551950    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" podUID="564888f8-35bb-4028-8040-110c959c9b19" containerName="main" probeResult="failure" output="Get \"https://10.132.0.42:8000/ready\": context deadline exceeded"
Oct 31 03:53:14.667293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.586698    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.667293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.640039    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.667293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.585404    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 31 03:53:14.667293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.659683    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" podUID="a9f9a022-fa1c-4f08-ac54-cb751f629547" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 31 03:53:14.667293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.640075    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" podUID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.667293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.660563    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:53:14.667293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.660613    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:53:14.667293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.660651    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:53:14.667293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.640003    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l" podUID="a5f198d8-39a2-4c72-87de-516e077d47bd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.667293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.662627    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:53:14.681720 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.681671    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" podUID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.682325 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.682301    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:53:14.687796 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.687769    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 31 03:53:14.688046 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.688020    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz" podUID="a65ad9de-668e-4600-8643-546a6ce94b99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 31 03:53:14.770434 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.770396    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:53:14.770743 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.770711    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm" podUID="136c00fa-e898-4bfd-9b48-9241fba6740e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.890430 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.890383    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59" podUID="4b319682-afe9-4edd-b9c0-46d7cb7a989c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.890952 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.890927    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:53:14.987481 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.987432    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" podUID="ade2ace8-0217-404c-9592-85576f16b085" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:14.988138 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:14.988115    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:53:15.127584 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.127542    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 31 03:53:15.175295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.158603    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc" podUID="71787759-a0b2-43c1-9939-acfc039ca1eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 31 03:53:15.175295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.152985    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:15.175295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.152940    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn" podUID="e20732a1-da5b-4e22-b861-6b03dfe62522" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:15.175295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.159302    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:53:15.204881 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.204463    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:15.376745 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.376647    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4" podUID="6fc591ce-8543-4590-9dd5-9002226418d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.32:8000/ready\": context deadline exceeded"
Oct 31 03:53:15.400665 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.377572    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j" podUID="f97017f6-52c4-4d27-9aa7-5045b9ac85ee" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/ready\": context deadline exceeded"
Oct 31 03:53:15.401180 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.401157    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:53:15.403893 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.403874    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:53:15.601336 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.546459    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 31 03:53:15.620292 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.613400    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" podUID="a9f9a022-fa1c-4f08-ac54-cb751f629547" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 31 03:53:15.666352 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.664395    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" podUID="564888f8-35bb-4028-8040-110c959c9b19" containerName="main" probeResult="failure" output="Get \"https://10.132.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:15.666352 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.664810    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg" podUID="067339e5-0df5-421d-8b14-471a9b1a9735" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Oct 31 03:53:15.666352 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.665082    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj" podUID="a5ad5303-15e7-422a-a144-5598d90f33db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.41:8000/ready\": context deadline exceeded"
Oct 31 03:53:15.666352 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.665366    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l" podUID="a5f198d8-39a2-4c72-87de-516e077d47bd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:15.666352 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.665393    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" podUID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:15.666352 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.665366    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:15.681346 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.675707    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 31 03:53:15.697524 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.694388    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz" podUID="a65ad9de-668e-4600-8643-546a6ce94b99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 31 03:53:15.894635 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.894593    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59" podUID="4b319682-afe9-4edd-b9c0-46d7cb7a989c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:15.969576 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.948707    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:53:15.969576 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.948770    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:53:15.997401 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:15.997361    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" podUID="ade2ace8-0217-404c-9592-85576f16b085" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:16.010294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.009362    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:53:16.027779 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.027735    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:53:16.050213 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.050173    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:53:16.106085 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.106059    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:53:16.134074 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.134047    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:53:16.147578 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.147544    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 31 03:53:16.147857 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.147823    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc" podUID="71787759-a0b2-43c1-9939-acfc039ca1eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 31 03:53:16.202291 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.173537    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:53:16.202291 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.186451    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l" podUID="a5f198d8-39a2-4c72-87de-516e077d47bd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:16.202291 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.196772    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:53:16.281359 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.274918    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:53:16.283538 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.283499    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:53:16.283694 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.283578    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:53:16.287142 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.287116    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:53:16.459703 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.459666    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 31 03:53:16.459965 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.459936    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" podUID="a9f9a022-fa1c-4f08-ac54-cb751f629547" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 31 03:53:16.677414 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.677330    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 31 03:53:16.677622 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:16.677600    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz" podUID="a65ad9de-668e-4600-8643-546a6ce94b99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 31 03:53:17.066185 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:17.066147    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 31 03:53:17.066374 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:17.066226    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc" podUID="71787759-a0b2-43c1-9939-acfc039ca1eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 31 03:53:17.458383 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:17.458339    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 31 03:53:17.458909 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:17.458405    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" podUID="a9f9a022-fa1c-4f08-ac54-cb751f629547" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 31 03:53:17.690357 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:17.689335    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 31 03:53:17.690357 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:17.689385    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz" podUID="a65ad9de-668e-4600-8643-546a6ce94b99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 31 03:53:18.071295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:18.070747    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:18.071295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:18.071184    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:18.072033 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:18.071999    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:53:18.463054 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:18.463026    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:18.463562 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:18.463388    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:18.464308 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:18.464277    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 03:53:18.676569 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:18.676536    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Oct 31 03:53:18.676750 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:18.676592    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz" podUID="a65ad9de-668e-4600-8643-546a6ce94b99" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Oct 31 03:53:19.679701 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:19.679675    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:19.680319 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:19.680259    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:19.681019 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:19.681001    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 03:53:23.689503 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:23.689475    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 03:53:23.690545 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:23.690524    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 03:53:35.312620 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:53:35.312577    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:53:44.198399 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:44.198352    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Oct 31 03:53:44.224293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:44.220454    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:44.401088 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:44.400977    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4" podUID="6fc591ce-8543-4590-9dd5-9002226418d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:44.427293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:44.425344    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" podUID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:53:44.587643 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:53:44.587573    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:05.314301 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:54:05.314250    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:54:08.204745 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.204706    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"]
Oct 31 03:54:08.208539 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.208513    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.295689 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.295654    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"]
Oct 31 03:54:08.331699 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.331672    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.331875 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.331724    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.331875 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.331845    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.331972 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.331882    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff3deeee-d307-4df0-8656-dcc3a13e060a-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.331972 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.331906    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thb9h\" (UniqueName: \"kubernetes.io/projected/ff3deeee-d307-4df0-8656-dcc3a13e060a-kube-api-access-thb9h\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.332048 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.332010    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.332048 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.332028    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.332124 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.332072    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.332171 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.332132    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.432952 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.432921    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.433140 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.432959    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.433140 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.432986    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.433140 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.433051    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.433371 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.433162    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.433371 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.433218    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.433371 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.433298    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.433371 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.433334    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff3deeee-d307-4df0-8656-dcc3a13e060a-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.433371 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.433359    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thb9h\" (UniqueName: \"kubernetes.io/projected/ff3deeee-d307-4df0-8656-dcc3a13e060a-kube-api-access-thb9h\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.433631 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.433403    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-credential-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.433631 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.433450    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-workload-certs\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.433631 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.433479    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-data\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.433797 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.433736    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-workload-socket\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.434047 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.434026    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff3deeee-d307-4df0-8656-dcc3a13e060a-istiod-ca-cert\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.435804 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.435785    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-envoy\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.436022 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.436006    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-podinfo\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.503097 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.503035    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-token\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.503207 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.503141    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thb9h\" (UniqueName: \"kubernetes.io/projected/ff3deeee-d307-4df0-8656-dcc3a13e060a-kube-api-access-thb9h\") pod \"e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") " pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.522043 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.522019    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:08.944089 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:54:08.944058    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff3deeee_d307_4df0_8656_dcc3a13e060a.slice/crio-7f0595c5198fd5df6c1b928776a3f61bb29c6fa3d34ea4169c35a67dd354a6d0 WatchSource:0}: Error finding container 7f0595c5198fd5df6c1b928776a3f61bb29c6fa3d34ea4169c35a67dd354a6d0: Status 404 returned error can't find the container with id 7f0595c5198fd5df6c1b928776a3f61bb29c6fa3d34ea4169c35a67dd354a6d0
Oct 31 03:54:08.946508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.946475    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:54:08.946616 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.946541    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:54:08.946616 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.946571    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:54:08.947495 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:08.947470    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"]
Oct 31 03:54:09.311224 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:09.311183    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7" event={"ID":"ff3deeee-d307-4df0-8656-dcc3a13e060a","Type":"ContainerStarted","Data":"7fe66428b9f5afbec05882f58f08d66ea5f1c0dda207bde5fe4e4273183993c0"}
Oct 31 03:54:09.311598 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:09.311299    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7" event={"ID":"ff3deeee-d307-4df0-8656-dcc3a13e060a","Type":"ContainerStarted","Data":"7f0595c5198fd5df6c1b928776a3f61bb29c6fa3d34ea4169c35a67dd354a6d0"}
Oct 31 03:54:09.435631 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:09.435579    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7" podStartSLOduration=1.435564807 podStartE2EDuration="1.435564807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:54:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:54:09.433817217 +0000 UTC m=+1546.019357478" watchObservedRunningTime="2026-10-31 03:54:09.435564807 +0000 UTC m=+1546.021105068"
Oct 31 03:54:09.522769 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:09.522739    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:10.534818 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:10.534781    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 31 03:54:10.535255 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:10.534845    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7" podUID="ff3deeee-d307-4df0-8656-dcc3a13e060a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 31 03:54:11.522675 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:11.522626    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 31 03:54:11.524260 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:11.524226    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7" podUID="ff3deeee-d307-4df0-8656-dcc3a13e060a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 31 03:54:12.532641 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:12.532607    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 31 03:54:12.533224 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:12.533192    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7" podUID="ff3deeee-d307-4df0-8656-dcc3a13e060a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 31 03:54:13.529290 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:13.528688    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 31 03:54:13.529290 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:13.528756    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7" podUID="ff3deeee-d307-4df0-8656-dcc3a13e060a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 31 03:54:14.526299 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:14.525611    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 31 03:54:14.526299 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:14.525686    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7" podUID="ff3deeee-d307-4df0-8656-dcc3a13e060a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 31 03:54:15.523437 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:15.523398    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 31 03:54:15.523683 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:15.523464    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7" podUID="ff3deeee-d307-4df0-8656-dcc3a13e060a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 31 03:54:16.526959 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:16.526927    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:16.527398 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:16.527205    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:16.527824 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:16.527808    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 03:54:19.027290 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.024422    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" podUID="ade2ace8-0217-404c-9592-85576f16b085" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:19.027290 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.025695    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn" podUID="e20732a1-da5b-4e22-b861-6b03dfe62522" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:19.027290 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.025995    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg" podUID="067339e5-0df5-421d-8b14-471a9b1a9735" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Oct 31 03:54:19.090294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.089359    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:19.125728 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.125688    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:54:19.126013 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.125982    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc" podUID="71787759-a0b2-43c1-9939-acfc039ca1eb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:19.173302 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.172472    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:19.242419 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.218368    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j" podUID="f97017f6-52c4-4d27-9aa7-5045b9ac85ee" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:19.242419 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.219256    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:19.399371 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.383396    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" podUID="ade2ace8-0217-404c-9592-85576f16b085" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:19.428754 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.428706    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4" podUID="6fc591ce-8543-4590-9dd5-9002226418d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:19.444876 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.428991    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" podUID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:19.671348 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.668351    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj" podUID="a5ad5303-15e7-422a-a144-5598d90f33db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:19.671348 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.668778    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l" podUID="a5f198d8-39a2-4c72-87de-516e077d47bd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Oct 31 03:54:19.671348 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.669932    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" podUID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:19.671348 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.670221    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" podUID="564888f8-35bb-4028-8040-110c959c9b19" containerName="main" probeResult="failure" output="Get \"https://10.132.0.42:8000/ready\": context deadline exceeded"
Oct 31 03:54:19.886294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:19.880595    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" podUID="564888f8-35bb-4028-8040-110c959c9b19" containerName="main" probeResult="failure" output="Get \"https://10.132.0.42:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:20.029844 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:20.029798    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn" podUID="e20732a1-da5b-4e22-b861-6b03dfe62522" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:20.030745 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:20.030723    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:54:20.084256 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:20.084227    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:54:35.315650 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:54:35.315596    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:54:39.065223 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.065183    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"]
Oct 31 03:54:39.069210 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.069192    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.158492 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.158423    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"]
Oct 31 03:54:39.207011 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.206960    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.207202 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.207120    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5125225-3fc9-4a55-8df9-0bce428ea38d-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.207283 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.207210    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.207283 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.207242    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.207392 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.207287    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.207440 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.207414    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.207493 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.207446    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.207544 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.207498    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9crl7\" (UniqueName: \"kubernetes.io/projected/d5125225-3fc9-4a55-8df9-0bce428ea38d-kube-api-access-9crl7\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.207608 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.207593    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.308979 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.308947    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.308979 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.308980    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.309234 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.309116    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9crl7\" (UniqueName: \"kubernetes.io/projected/d5125225-3fc9-4a55-8df9-0bce428ea38d-kube-api-access-9crl7\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.309234 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.309173    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.309376 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.309316    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.309376 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.309348    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.309484 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.309389    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.309484 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.309453    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5125225-3fc9-4a55-8df9-0bce428ea38d-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.309595 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.309562    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.309651 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.309602    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.309651 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.309612    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.309760 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.309668    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.309863 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.309840    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.310116 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.310096    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5125225-3fc9-4a55-8df9-0bce428ea38d-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.311840 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.311813    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.311958 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.311939    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.377960 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.377899    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9crl7\" (UniqueName: \"kubernetes.io/projected/d5125225-3fc9-4a55-8df9-0bce428ea38d-kube-api-access-9crl7\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.379217 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.379195    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.382200 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.382184    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:39.811902 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:54:39.811874    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5125225_3fc9_4a55_8df9_0bce428ea38d.slice/crio-a83dedc60e0769728d674742eb5c3fbe56190528fc5827c2be461e68808c4d3f WatchSource:0}: Error finding container a83dedc60e0769728d674742eb5c3fbe56190528fc5827c2be461e68808c4d3f: Status 404 returned error can't find the container with id a83dedc60e0769728d674742eb5c3fbe56190528fc5827c2be461e68808c4d3f
Oct 31 03:54:39.814527 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.814487    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:54:39.814662 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.814570    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:54:39.814662 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.814632    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:54:39.821479 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.817636    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"]
Oct 31 03:54:39.838822 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.838796    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"]
Oct 31 03:54:39.843806 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.843789    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:39.923388 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:39.923352    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"]
Oct 31 03:54:40.019348 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.019320    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.019542 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.019369    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.019542 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.019461    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37196bbd-c341-4697-89c0-04ccffdfec09-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.019542 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.019517    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft25p\" (UniqueName: \"kubernetes.io/projected/37196bbd-c341-4697-89c0-04ccffdfec09-kube-api-access-ft25p\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.019707 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.019618    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.019707 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.019666    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.019870 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.019822    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.019921 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.019871    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37196bbd-c341-4697-89c0-04ccffdfec09-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.020011 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.019989    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37196bbd-c341-4697-89c0-04ccffdfec09-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.024382 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.024360    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"]
Oct 31 03:54:40.028341 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.028322    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.096813 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.096734    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"]
Oct 31 03:54:40.120962 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.120930    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.120962 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.120964    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37196bbd-c341-4697-89c0-04ccffdfec09-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.121188 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121098    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft25p\" (UniqueName: \"kubernetes.io/projected/37196bbd-c341-4697-89c0-04ccffdfec09-kube-api-access-ft25p\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.121188 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121170    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.121322 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121219    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.121322 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121303    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.121424 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121373    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbwwk\" (UniqueName: \"kubernetes.io/projected/8215d8db-0143-4917-97a5-091ba6bd2f6c-kube-api-access-pbwwk\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.121518 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121497    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.121581 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121530    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8215d8db-0143-4917-97a5-091ba6bd2f6c-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.121581 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121556    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.121581 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121565    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.121734 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121655    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.121734 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121693    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37196bbd-c341-4697-89c0-04ccffdfec09-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.121835 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121755    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.121835 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121733    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.121937 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121783    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37196bbd-c341-4697-89c0-04ccffdfec09-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.122002 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121944    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.122002 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.121990    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.122111 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.122060    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.122111 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.122099    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.122222 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.122128    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.122342 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.122323    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.122514 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.122492    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37196bbd-c341-4697-89c0-04ccffdfec09-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.123563 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.123539    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.124216 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.124192    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37196bbd-c341-4697-89c0-04ccffdfec09-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.183929 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.183891    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37196bbd-c341-4697-89c0-04ccffdfec09-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.184079 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.183978    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft25p\" (UniqueName: \"kubernetes.io/projected/37196bbd-c341-4697-89c0-04ccffdfec09-kube-api-access-ft25p\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.223809 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.223778    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.223990 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.223821    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.223990 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.223915    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.223990 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.223952    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbwwk\" (UniqueName: \"kubernetes.io/projected/8215d8db-0143-4917-97a5-091ba6bd2f6c-kube-api-access-pbwwk\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.224167 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.223994    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8215d8db-0143-4917-97a5-091ba6bd2f6c-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.224167 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.224117    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.224335 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.224172    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.224335 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.224193    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-data\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.224475 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.224378    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.224536 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.224481    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-credential-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.224654 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.224632    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8215d8db-0143-4917-97a5-091ba6bd2f6c-istiod-ca-cert\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.224862 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.224841    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-workload-certs\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.225006 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.224984    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.225327 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.225307    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-workload-socket\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.226414 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.226392    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-envoy\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.227035 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.227019    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-podinfo\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.287202 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.287173    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-token\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.287384 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.287238    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbwwk\" (UniqueName: \"kubernetes.io/projected/8215d8db-0143-4917-97a5-091ba6bd2f6c-kube-api-access-pbwwk\") pod \"e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") " pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.345955 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.345920    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:40.463721 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.463692    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:40.586693 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.585849    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj" event={"ID":"d5125225-3fc9-4a55-8df9-0bce428ea38d","Type":"ContainerStarted","Data":"87dcb492a8877a5ca035962fbc223c69bf33e58110e90ee2ce3fc5e107fa5245"}
Oct 31 03:54:40.586693 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.585931    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj" event={"ID":"d5125225-3fc9-4a55-8df9-0bce428ea38d","Type":"ContainerStarted","Data":"a83dedc60e0769728d674742eb5c3fbe56190528fc5827c2be461e68808c4d3f"}
Oct 31 03:54:40.730292 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.711823    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj" podStartSLOduration=1.711803732 podStartE2EDuration="1.711803732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:54:40.709828759 +0000 UTC m=+1577.295369022" watchObservedRunningTime="2026-10-31 03:54:40.711803732 +0000 UTC m=+1577.297343995"
Oct 31 03:54:40.798822 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:54:40.798791    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8215d8db_0143_4917_97a5_091ba6bd2f6c.slice/crio-152d668e67da1d958ae6314874d2446fe0752e623bccf69da33dec6540aec9e8 WatchSource:0}: Error finding container 152d668e67da1d958ae6314874d2446fe0752e623bccf69da33dec6540aec9e8: Status 404 returned error can't find the container with id 152d668e67da1d958ae6314874d2446fe0752e623bccf69da33dec6540aec9e8
Oct 31 03:54:40.801726 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.801690    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:54:40.801814 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.801776    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:54:40.801860 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.801820    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:54:40.803767 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.803743    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"]
Oct 31 03:54:40.874343 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:54:40.874317    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37196bbd_c341_4697_89c0_04ccffdfec09.slice/crio-7667813a4ce3cb3aba10d83e2e41dd13db7ac3a42b31ded36c640f6d0fdfef9f WatchSource:0}: Error finding container 7667813a4ce3cb3aba10d83e2e41dd13db7ac3a42b31ded36c640f6d0fdfef9f: Status 404 returned error can't find the container with id 7667813a4ce3cb3aba10d83e2e41dd13db7ac3a42b31ded36c640f6d0fdfef9f
Oct 31 03:54:40.877388 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.877365    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"]
Oct 31 03:54:40.877544 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.877515    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:54:40.877700 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.877577    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:54:40.877700 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:40.877610    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:54:41.383287 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.383239    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:41.384671 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.384645    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 31 03:54:41.384814 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.384697    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj" podUID="d5125225-3fc9-4a55-8df9-0bce428ea38d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 31 03:54:41.388035 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.388012    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6965-da4ba7"]
Oct 31 03:54:41.392738 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.392719    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6965-da4ba7"
Oct 31 03:54:41.450027 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.449950    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6965-da4ba7"]
Oct 31 03:54:41.454472 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.454446    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rgmz\" (UniqueName: \"kubernetes.io/projected/df80943a-29e4-4132-a506-c8e96b6932ad-kube-api-access-8rgmz\") pod \"test-curl-6965-da4ba7\" (UID: \"df80943a-29e4-4132-a506-c8e96b6932ad\") " pod="opendatahub/test-curl-6965-da4ba7"
Oct 31 03:54:41.496281 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.496236    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-2x2q5\""
Oct 31 03:54:41.556215 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.555874    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rgmz\" (UniqueName: \"kubernetes.io/projected/df80943a-29e4-4132-a506-c8e96b6932ad-kube-api-access-8rgmz\") pod \"test-curl-6965-da4ba7\" (UID: \"df80943a-29e4-4132-a506-c8e96b6932ad\") " pod="opendatahub/test-curl-6965-da4ba7"
Oct 31 03:54:41.594850 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.594201    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp" event={"ID":"37196bbd-c341-4697-89c0-04ccffdfec09","Type":"ContainerStarted","Data":"8199d38beedaac2403f85bd8805366b90f91e0c8bd8b4f9c8e249f1dc7f4f607"}
Oct 31 03:54:41.594850 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.594284    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp" event={"ID":"37196bbd-c341-4697-89c0-04ccffdfec09","Type":"ContainerStarted","Data":"7667813a4ce3cb3aba10d83e2e41dd13db7ac3a42b31ded36c640f6d0fdfef9f"}
Oct 31 03:54:41.598033 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.598011    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84" event={"ID":"8215d8db-0143-4917-97a5-091ba6bd2f6c","Type":"ContainerStarted","Data":"c8b8e1b5bdd4f00bc32f20408fa2755a3ed00383a9693952759e4a98fdba08e2"}
Oct 31 03:54:41.598208 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.598195    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84" event={"ID":"8215d8db-0143-4917-97a5-091ba6bd2f6c","Type":"ContainerStarted","Data":"152d668e67da1d958ae6314874d2446fe0752e623bccf69da33dec6540aec9e8"}
Oct 31 03:54:41.672340 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.668842    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rgmz\" (UniqueName: \"kubernetes.io/projected/df80943a-29e4-4132-a506-c8e96b6932ad-kube-api-access-8rgmz\") pod \"test-curl-6965-da4ba7\" (UID: \"df80943a-29e4-4132-a506-c8e96b6932ad\") " pod="opendatahub/test-curl-6965-da4ba7"
Oct 31 03:54:41.707004 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.706969    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6965-da4ba7"
Oct 31 03:54:41.965570 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.938670    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp" podStartSLOduration=2.938645779 podStartE2EDuration="2.938645779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:54:41.790704199 +0000 UTC m=+1578.376244464" watchObservedRunningTime="2026-10-31 03:54:41.938645779 +0000 UTC m=+1578.524186044"
Oct 31 03:54:41.998604 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:41.966722    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84" podStartSLOduration=2.966700432 podStartE2EDuration="2.966700432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:54:41.941663892 +0000 UTC m=+1578.527204166" watchObservedRunningTime="2026-10-31 03:54:41.966700432 +0000 UTC m=+1578.552240695"
Oct 31 03:54:42.356350 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:42.356081    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:42.356350 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:42.356208    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:54:42.356350 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:42.356250    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84" podUID="8215d8db-0143-4917-97a5-091ba6bd2f6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 31 03:54:42.383306 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:42.382710    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 31 03:54:42.383306 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:42.382777    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj" podUID="d5125225-3fc9-4a55-8df9-0bce428ea38d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 31 03:54:42.477371 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:42.477337    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:42.493930 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:42.492416    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 31 03:54:42.493930 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:42.492461    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp" podUID="37196bbd-c341-4697-89c0-04ccffdfec09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 31 03:54:42.494930 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:42.494903    4926 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 03:54:42.495057 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:42.494976    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6965-da4ba7"]
Oct 31 03:54:42.604081 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:42.604023    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6965-da4ba7" event={"ID":"df80943a-29e4-4132-a506-c8e96b6932ad","Type":"ContainerStarted","Data":"ddf88de0be88a40aee1432e436986d0a6ba3dd0b92b9ebded532c8d0a94373de"}
Oct 31 03:54:43.364317 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:43.356696    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:54:43.364317 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:43.356769    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84" podUID="8215d8db-0143-4917-97a5-091ba6bd2f6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 31 03:54:43.387338 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:43.386816    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 31 03:54:43.387338 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:43.386876    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj" podUID="d5125225-3fc9-4a55-8df9-0bce428ea38d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 31 03:54:43.469294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:43.468749    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 31 03:54:43.469294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:43.468810    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp" podUID="37196bbd-c341-4697-89c0-04ccffdfec09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 31 03:54:44.357293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:44.356833    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:54:44.357293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:44.356905    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84" podUID="8215d8db-0143-4917-97a5-091ba6bd2f6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 31 03:54:44.447293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:44.446409    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 31 03:54:44.447293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:44.446493    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj" podUID="d5125225-3fc9-4a55-8df9-0bce428ea38d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 31 03:54:44.523293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:44.507231    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 31 03:54:44.625350 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:44.603950    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp" podUID="37196bbd-c341-4697-89c0-04ccffdfec09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 31 03:54:45.346650 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:45.346601    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:54:45.346950 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:45.346915    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84" podUID="8215d8db-0143-4917-97a5-091ba6bd2f6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 31 03:54:45.400347 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:45.399874    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 31 03:54:45.400347 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:45.399930    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj" podUID="d5125225-3fc9-4a55-8df9-0bce428ea38d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 31 03:54:45.494310 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:45.481411    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 31 03:54:45.494310 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:45.481505    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp" podUID="37196bbd-c341-4697-89c0-04ccffdfec09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 31 03:54:46.351134 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:46.350682    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:54:46.351134 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:46.350754    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84" podUID="8215d8db-0143-4917-97a5-091ba6bd2f6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 31 03:54:46.383508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:46.383386    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 31 03:54:46.383508 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:46.383445    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj" podUID="d5125225-3fc9-4a55-8df9-0bce428ea38d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 31 03:54:46.465906 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:46.465791    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 31 03:54:46.465906 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:46.465848    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp" podUID="37196bbd-c341-4697-89c0-04ccffdfec09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 31 03:54:47.350857 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:47.350821    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:54:47.368860 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:47.368729    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84" podUID="8215d8db-0143-4917-97a5-091ba6bd2f6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 31 03:54:47.383643 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:47.383600    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 31 03:54:47.383838 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:47.383677    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj" podUID="d5125225-3fc9-4a55-8df9-0bce428ea38d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 31 03:54:47.467775 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:47.467471    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 31 03:54:47.467775 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:47.467534    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp" podUID="37196bbd-c341-4697-89c0-04ccffdfec09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 31 03:54:48.346633 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:48.346589    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Oct 31 03:54:48.346838 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:48.346673    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84" podUID="8215d8db-0143-4917-97a5-091ba6bd2f6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Oct 31 03:54:48.395020 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:48.394987    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:48.396251 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:48.396227    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:48.397185 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:48.397164    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 03:54:48.464974 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:48.464927    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 31 03:54:48.465183 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:48.465000    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp" podUID="37196bbd-c341-4697-89c0-04ccffdfec09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 31 03:54:49.401916 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:49.401884    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:49.402676 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:49.402590    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:49.403622 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:49.403576    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 03:54:49.469841 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:49.469807    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:49.470429 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:49.470405    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:49.471539 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:49.471519    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 03:54:49.866311 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:49.864506    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6965-da4ba7" event={"ID":"df80943a-29e4-4132-a506-c8e96b6932ad","Type":"ContainerStarted","Data":"4ca38787c984ccf6f36440f627b7dcb883d6dccaed060c688d48356fa0c787e8"}
Oct 31 03:54:49.982673 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:49.982583    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6965-da4ba7" podStartSLOduration=1.784942707 podStartE2EDuration="8.982568719s" totalImagesPullingTime="7.197626012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:54:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:54:49.979432217 +0000 UTC m=+1586.564972479" watchObservedRunningTime="2026-10-31 03:54:49.982568719 +0000 UTC m=+1586.568108981"
Oct 31 03:54:51.585084 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:51.585042    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6965-da4ba7"]
Oct 31 03:54:51.589754 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:51.589728    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6965-da4ba7"]
Oct 31 03:54:51.876068 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:51.875963    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6965-da4ba7" podUID="df80943a-29e4-4132-a506-c8e96b6932ad" containerName="test-curl-6965-da4ba7" containerID="cri-o://4ca38787c984ccf6f36440f627b7dcb883d6dccaed060c688d48356fa0c787e8" gracePeriod=2
Oct 31 03:54:51.934738 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:54:51.934123    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6965-da4ba7\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="df80943a-29e4-4132-a506-c8e96b6932ad" pod="opendatahub/test-curl-6965-da4ba7"
Oct 31 03:54:53.586962 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:54:53.586919    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6965-da4ba7\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="df80943a-29e4-4132-a506-c8e96b6932ad" pod="opendatahub/test-curl-6965-da4ba7"
Oct 31 03:54:54.138350 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.138292    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6965-da4ba7"
Oct 31 03:54:54.337588 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.337554    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df80943a-29e4-4132-a506-c8e96b6932ad-kube-api-access-8rgmz\" (UniqueName: \"kubernetes.io/projected/df80943a-29e4-4132-a506-c8e96b6932ad-kube-api-access-8rgmz\") pod \"df80943a-29e4-4132-a506-c8e96b6932ad\" (UID: \"df80943a-29e4-4132-a506-c8e96b6932ad\") "
Oct 31 03:54:54.342242 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.342209    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df80943a-29e4-4132-a506-c8e96b6932ad-kube-api-access-8rgmz" pod "df80943a-29e4-4132-a506-c8e96b6932ad" (UID: "df80943a-29e4-4132-a506-c8e96b6932ad"). InnerVolumeSpecName "kube-api-access-8rgmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:54:54.439490 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.439387    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rgmz\" (UniqueName: \"kubernetes.io/projected/df80943a-29e4-4132-a506-c8e96b6932ad-kube-api-access-8rgmz\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:54:54.898739 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.898698    4926 generic.go:356] "Generic (PLEG): container finished" podID="df80943a-29e4-4132-a506-c8e96b6932ad" containerID="4ca38787c984ccf6f36440f627b7dcb883d6dccaed060c688d48356fa0c787e8" exitCode=137
Oct 31 03:54:54.899226 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.898758    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6965-da4ba7"
Oct 31 03:54:54.899226 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.898851    4926 scope.go:122] "RemoveContainer" containerID="4ca38787c984ccf6f36440f627b7dcb883d6dccaed060c688d48356fa0c787e8"
Oct 31 03:54:54.910242 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.910218    4926 scope.go:122] "RemoveContainer" containerID="4ca38787c984ccf6f36440f627b7dcb883d6dccaed060c688d48356fa0c787e8"
Oct 31 03:54:54.910585 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:54:54.910565    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ca38787c984ccf6f36440f627b7dcb883d6dccaed060c688d48356fa0c787e8\": container with ID starting with 4ca38787c984ccf6f36440f627b7dcb883d6dccaed060c688d48356fa0c787e8 not found: ID does not exist" containerID="4ca38787c984ccf6f36440f627b7dcb883d6dccaed060c688d48356fa0c787e8"
Oct 31 03:54:54.910642 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.910595    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ca38787c984ccf6f36440f627b7dcb883d6dccaed060c688d48356fa0c787e8"} err="failed to get container status \"4ca38787c984ccf6f36440f627b7dcb883d6dccaed060c688d48356fa0c787e8\": rpc error: code = NotFound desc = could not find container \"4ca38787c984ccf6f36440f627b7dcb883d6dccaed060c688d48356fa0c787e8\": container with ID starting with 4ca38787c984ccf6f36440f627b7dcb883d6dccaed060c688d48356fa0c787e8 not found: ID does not exist"
Oct 31 03:54:54.927788 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.927764    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"]
Oct 31 03:54:54.928334 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.928320    4926 cpu_manager.go:414] "containerMap: removing container" podUID="df80943a-29e4-4132-a506-c8e96b6932ad" containerName="test-curl-6965-da4ba7"
Oct 31 03:54:54.928381 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.928336    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df80943a-29e4-4132-a506-c8e96b6932ad" containerName="test-curl-6965-da4ba7"
Oct 31 03:54:54.928466 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.928456    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="df80943a-29e4-4132-a506-c8e96b6932ad" containerName="test-curl-6965-da4ba7"
Oct 31 03:54:54.933208 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.933188    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:54.944308 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.944284    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:54.944433 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.944340    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:54.944433 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.944413    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:54.944562 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.944443    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bfk8\" (UniqueName: \"kubernetes.io/projected/d41233ae-9c3e-466a-aa39-075d14ba5e0f-kube-api-access-5bfk8\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:54.944562 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.944462    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:54.944675 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.944575    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:54.944675 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.944623    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:54.944785 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.944698    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:54.944785 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.944743    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:54.961468 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:54:54.961441    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6965-da4ba7\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="df80943a-29e4-4132-a506-c8e96b6932ad" pod="opendatahub/test-curl-6965-da4ba7"
Oct 31 03:54:54.995745 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:54.995713    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"]
Oct 31 03:54:55.015397 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:54:55.015357    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6965-da4ba7\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="df80943a-29e4-4132-a506-c8e96b6932ad" pod="opendatahub/test-curl-6965-da4ba7"
Oct 31 03:54:55.045959 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.045923    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.045959 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.045955    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.046206 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.045994    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.046206 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.046021    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bfk8\" (UniqueName: \"kubernetes.io/projected/d41233ae-9c3e-466a-aa39-075d14ba5e0f-kube-api-access-5bfk8\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.046206 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.046040    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.046485 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.046360    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-credential-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.046578 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.046555    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.046646 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.046613    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-workload-certs\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.046717 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.046659    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.046717 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.046711    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.046820 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.046766    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.047091 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.047063    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-workload-socket\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.047178 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.047098    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-data\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.047325 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.047297    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istiod-ca-cert\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.049297 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.049250    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-envoy\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.049752 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.049731    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-podinfo\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.111053 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.111019    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-token\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.111225 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.111151    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bfk8\" (UniqueName: \"kubernetes.io/projected/d41233ae-9c3e-466a-aa39-075d14ba5e0f-kube-api-access-5bfk8\") pod \"e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") " pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.246478 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.246447    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:55.538724 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.538651    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df80943a-29e4-4132-a506-c8e96b6932ad" path="/var/lib/kubelet/pods/df80943a-29e4-4132-a506-c8e96b6932ad/volumes"
Oct 31 03:54:55.677173 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.676888    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"]
Oct 31 03:54:55.678036 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.677997    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:54:55.678152 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.678104    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:54:55.678203 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.678150    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:54:55.938899 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.938794    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" event={"ID":"d41233ae-9c3e-466a-aa39-075d14ba5e0f","Type":"ContainerStarted","Data":"86ace4b78496d8eb24077904d2a339dbc61dd08c60096983dcc94a10ed3a35d5"}
Oct 31 03:54:55.938899 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:55.938864    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" event={"ID":"d41233ae-9c3e-466a-aa39-075d14ba5e0f","Type":"ContainerStarted","Data":"dc476f85b109d60d12def75090137c12b343d569e73bffb3b5e31717a26bb42f"}
Oct 31 03:54:56.069931 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:56.069531    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" podStartSLOduration=2.069511378 podStartE2EDuration="2.069511378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:54:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:54:56.068513176 +0000 UTC m=+1592.654053444" watchObservedRunningTime="2026-10-31 03:54:56.069511378 +0000 UTC m=+1592.655051641"
Oct 31 03:54:56.246806 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:56.246770    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:54:56.248147 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:56.248123    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:54:56.248289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:56.248177    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 31 03:54:57.251680 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:57.251647    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:54:57.252155 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:57.252132    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 31 03:54:58.251292 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:58.250713    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:54:58.251292 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:58.250795    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 31 03:54:58.901295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:58.898219    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59" podUID="4b319682-afe9-4edd-b9c0-46d7cb7a989c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:59.011295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.010482    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" podUID="ade2ace8-0217-404c-9592-85576f16b085" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Oct 31 03:54:59.036294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.031410    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg" podUID="067339e5-0df5-421d-8b14-471a9b1a9735" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:59.140295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.139761    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn" podUID="e20732a1-da5b-4e22-b861-6b03dfe62522" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:59.301293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.300381    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Oct 31 03:54:59.301293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.300800    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Oct 31 03:54:59.329295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.328835    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j" podUID="f97017f6-52c4-4d27-9aa7-5045b9ac85ee" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:59.353296 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.351325    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:54:59.353296 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.351992    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 31 03:54:59.389296 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.388845    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Oct 31 03:54:59.511865 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.491981    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4" podUID="6fc591ce-8543-4590-9dd5-9002226418d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.32:8000/ready\": context deadline exceeded"
Oct 31 03:54:59.513489 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.512333    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" podUID="ade2ace8-0217-404c-9592-85576f16b085" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/health\": context deadline exceeded"
Oct 31 03:54:59.513489 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.512449    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" podUID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Oct 31 03:54:59.668119 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.668016    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l" podUID="a5f198d8-39a2-4c72-87de-516e077d47bd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Oct 31 03:54:59.704324 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.674710    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" podUID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:59.741245 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.674849    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" podUID="564888f8-35bb-4028-8040-110c959c9b19" containerName="main" probeResult="failure" output="Get \"https://10.132.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:54:59.741245 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.674946    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj" podUID="a5ad5303-15e7-422a-a144-5598d90f33db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.41:8000/ready\": context deadline exceeded"
Oct 31 03:54:59.924349 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.924232    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59" podUID="4b319682-afe9-4edd-b9c0-46d7cb7a989c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:00.044155 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:54:59.952388    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" podUID="564888f8-35bb-4028-8040-110c959c9b19" containerName="main" probeResult="failure" output="Get \"https://10.132.0.42:8000/health\": context deadline exceeded"
Oct 31 03:55:00.044426 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.018646    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" podUID="ade2ace8-0217-404c-9592-85576f16b085" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:00.068293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.045594    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:55:00.068606 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.068585    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:55:00.070297 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.061210    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg" podUID="067339e5-0df5-421d-8b14-471a9b1a9735" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Oct 31 03:55:00.073760 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.073735    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:55:00.149300 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.141364    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn" podUID="e20732a1-da5b-4e22-b861-6b03dfe62522" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:00.172295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.150089    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:55:00.302289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.302225    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:55:00.302601 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.302554    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 31 03:55:00.302832 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.302812    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Oct 31 03:55:00.304279 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.304238    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:55:00.363969 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.338663    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j" podUID="f97017f6-52c4-4d27-9aa7-5045b9ac85ee" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/ready\": context deadline exceeded"
Oct 31 03:55:00.375715 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.364330    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:55:00.375715 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.367069    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Oct 31 03:55:00.376182 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.376139    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:55:00.422299 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.405339    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:00.422299 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.405747    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:55:00.539291 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.522986    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" podUID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:00.594704 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.594618    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:55:00.632295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.628534    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4" podUID="6fc591ce-8543-4590-9dd5-9002226418d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:00.645294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.634003    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:55:00.710294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.687509    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l" podUID="a5f198d8-39a2-4c72-87de-516e077d47bd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Oct 31 03:55:00.710294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.688796    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:55:00.711208 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.711170    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" podUID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:00.734893 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.717325    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj" podUID="a5ad5303-15e7-422a-a144-5598d90f33db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:00.735314 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.717361    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" podUID="564888f8-35bb-4028-8040-110c959c9b19" containerName="main" probeResult="failure" output="Get \"https://10.132.0.42:8000/ready\": context deadline exceeded"
Oct 31 03:55:00.758378 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.735193    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:55:00.758378 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.735597    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:55:00.758378 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.735634    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:55:01.044297 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.973782    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:55:01.044297 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:00.978959    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:01.086289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.045354    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" podUID="ade2ace8-0217-404c-9592-85576f16b085" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Oct 31 03:55:01.086289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.045878    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59" podUID="4b319682-afe9-4edd-b9c0-46d7cb7a989c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Oct 31 03:55:01.086289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.070404    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" podUID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:01.086289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.072350    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg" podUID="067339e5-0df5-421d-8b14-471a9b1a9735" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Oct 31 03:55:01.160921 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.160891    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:55:01.189004 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.187653    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 03:55:01.204694 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.189740    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn" podUID="e20732a1-da5b-4e22-b861-6b03dfe62522" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:01.359356 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.303047    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:55:01.446295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.388237    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 31 03:55:01.446295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.344140    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j" podUID="f97017f6-52c4-4d27-9aa7-5045b9ac85ee" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:01.446295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.387612    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59" podUID="4b319682-afe9-4edd-b9c0-46d7cb7a989c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:01.446295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.308159    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Oct 31 03:55:01.446295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.344072    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:55:01.446295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.407240    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj" podUID="2524a945-cbb0-42b3-b2be-8cd43a1f1df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:01.446295 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.407372    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:01.523226 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.523176    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" podUID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:01.681353 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.626379    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4" podUID="6fc591ce-8543-4590-9dd5-9002226418d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.32:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:01.791666 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.640366    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4" podUID="6fc591ce-8543-4590-9dd5-9002226418d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:01.792490 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.739418    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" podUID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:01.814179 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.787467    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj" podUID="a5ad5303-15e7-422a-a144-5598d90f33db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:01.814179 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.789135    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" podUID="564888f8-35bb-4028-8040-110c959c9b19" containerName="main" probeResult="failure" output="Get \"https://10.132.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:01.974418 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:01.974360    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.056705 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.056376    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" podUID="ade2ace8-0217-404c-9592-85576f16b085" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.143555 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.143418    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:55:02.143555 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.143485    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448" podUID="69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.153176 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.153137    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg" podUID="067339e5-0df5-421d-8b14-471a9b1a9735" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Oct 31 03:55:02.160771 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.160734    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l" podUID="a5f198d8-39a2-4c72-87de-516e077d47bd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.191563 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.190380    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn" podUID="e20732a1-da5b-4e22-b861-6b03dfe62522" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.266349 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.249953    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:55:02.266349 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.250013    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 31 03:55:02.413770 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.411767    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.413770 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.412690    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j" podUID="f97017f6-52c4-4d27-9aa7-5045b9ac85ee" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/ready\": context deadline exceeded"
Oct 31 03:55:02.413770 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.413742    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.422926 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.417775    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:55:02.422926 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.417837    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5" podUID="a977c9a1-fe79-4f13-8aa0-9a3484a28238" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.447503 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.439568    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:55:02.447503 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.439633    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp" podUID="9bdad2ce-6f22-4a30-b7e2-328bededd05b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.600349 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.533392    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" podUID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.793546 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.793494    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4" podUID="6fc591ce-8543-4590-9dd5-9002226418d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:02.810305 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.799581    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" podUID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": context deadline exceeded"
Oct 31 03:55:02.835304 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.825593    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" podUID="564888f8-35bb-4028-8040-110c959c9b19" containerName="main" probeResult="failure" output="Get \"https://10.132.0.42:8000/ready\": context deadline exceeded"
Oct 31 03:55:02.851350 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.836353    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj" podUID="a5ad5303-15e7-422a-a144-5598d90f33db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.41:8000/ready\": context deadline exceeded"
Oct 31 03:55:03.001309 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.992598    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:03.001309 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:02.992969    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:55:03.163651 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.163520    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59" podUID="4b319682-afe9-4edd-b9c0-46d7cb7a989c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Oct 31 03:55:03.194360 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.187351    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l" podUID="a5f198d8-39a2-4c72-87de-516e077d47bd" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Oct 31 03:55:03.194360 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.189024    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg" podUID="067339e5-0df5-421d-8b14-471a9b1a9735" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Oct 31 03:55:03.194360 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.189621    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:55:03.374307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.313310    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 03:55:03.374307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.315469    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:55:03.374307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.315526    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 31 03:55:03.374307 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.326026    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" podUID="ade2ace8-0217-404c-9592-85576f16b085" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:03.448430 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.422217    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j" podUID="f97017f6-52c4-4d27-9aa7-5045b9ac85ee" containerName="main" probeResult="failure" output="Get \"https://10.132.0.40:8000/ready\": context deadline exceeded"
Oct 31 03:55:03.541338 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.506156    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Oct 31 03:55:03.598043 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.597995    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" podUID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:03.773618 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.773581    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Oct 31 03:55:03.824765 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.824723    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4" podUID="6fc591ce-8543-4590-9dd5-9002226418d2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.32:8000/ready\": context deadline exceeded"
Oct 31 03:55:03.846790 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.846745    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" podUID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" containerName="main" probeResult="failure" output="Get \"https://10.132.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:03.856228 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.840363    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" podUID="564888f8-35bb-4028-8040-110c959c9b19" containerName="main" probeResult="failure" output="Get \"https://10.132.0.42:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:03.856824 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.856489    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj" podUID="a5ad5303-15e7-422a-a144-5598d90f33db" containerName="main" probeResult="failure" output="Get \"https://10.132.0.41:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:03.952850 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.952808    4926 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:03.964904 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.964874    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 03:55:03.965370 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.965355    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 03:55:03.965718 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.965509    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 03:55:03.973460 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.973382    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 03:55:03.976032 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.976009    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 03:55:03.981895 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.979457    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 03:55:03.996801 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.994820    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 03:55:03.996801 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.995804    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 03:55:03.996801 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.996246    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 03:55:03.998660 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.998498    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 03:55:03.998872 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:03.998855    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 03:55:04.001010 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:04.000988    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4"
Oct 31 03:55:04.246745 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:04.246710    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:55:04.247193 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:04.246770    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 31 03:55:05.247034 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:05.246991    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 31 03:55:05.247462 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:05.247054    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 31 03:55:05.325842 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:55:05.325803    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:55:07.288414 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:07.288381    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:55:07.288843 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:07.288431    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:55:07.376172 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:07.376145    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:55:07.376529 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:07.376504    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:55:07.420969 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:07.420944    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 03:55:13.252637 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.252602    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"]
Oct 31 03:55:13.256896 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.256874    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.310904 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.310869    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 31 03:55:13.310904 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.310885    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 31 03:55:13.311129 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.311018    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 31 03:55:13.311129 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.311057    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bfz2c\""
Oct 31 03:55:13.367696 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.367661    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"]
Oct 31 03:55:13.406569 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.406537    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.406759 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.406691    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.406759 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.406736    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.406914 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.406900    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.406976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.406960    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d0301a0-5d7b-4305-b4dd-24539a745220-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.407030 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.407011    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkpzh\" (UniqueName: \"kubernetes.io/projected/7d0301a0-5d7b-4305-b4dd-24539a745220-kube-api-access-gkpzh\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.407139 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.407118    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.508012 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.507927    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.508012 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.507980    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d0301a0-5d7b-4305-b4dd-24539a745220-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.508226 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.508024    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkpzh\" (UniqueName: \"kubernetes.io/projected/7d0301a0-5d7b-4305-b4dd-24539a745220-kube-api-access-gkpzh\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.508226 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.508157    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.508356 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.508238    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.508417 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.508396    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.508466 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.508447    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.509194 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.509162    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.510624 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.510586    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d0301a0-5d7b-4305-b4dd-24539a745220-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.510742 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.510694    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.510780 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.510750    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.511141 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.511126    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.570783 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.570746    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.571032 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.571012    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkpzh\" (UniqueName: \"kubernetes.io/projected/7d0301a0-5d7b-4305-b4dd-24539a745220-kube-api-access-gkpzh\") pod \"istiod-openshift-gateway-7ddd9cfdc8-6ndmq\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:13.869458 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:13.869379    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:14.301398 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:55:14.301369    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d0301a0_5d7b_4305_b4dd_24539a745220.slice/crio-f91573770c7842735df02f09c70c74d7c3484abdbd6b93b7a9e4c670a1c2c078 WatchSource:0}: Error finding container f91573770c7842735df02f09c70c74d7c3484abdbd6b93b7a9e4c670a1c2c078: Status 404 returned error can't find the container with id f91573770c7842735df02f09c70c74d7c3484abdbd6b93b7a9e4c670a1c2c078
Oct 31 03:55:14.304245 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:14.304211    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:55:14.304364 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:14.304325    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:55:14.305305 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:14.305282    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"]
Oct 31 03:55:14.951925 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:14.951885    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq" event={"ID":"7d0301a0-5d7b-4305-b4dd-24539a745220","Type":"ContainerStarted","Data":"e0effab5bf975bc5912bd7155f1c8b4d735adbd51adf4b16712109aa995a4845"}
Oct 31 03:55:14.952131 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:14.951939    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq" event={"ID":"7d0301a0-5d7b-4305-b4dd-24539a745220","Type":"ContainerStarted","Data":"f91573770c7842735df02f09c70c74d7c3484abdbd6b93b7a9e4c670a1c2c078"}
Oct 31 03:55:14.952131 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:14.952005    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:15.181131 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:15.181082    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq" podStartSLOduration=2.181070063 podStartE2EDuration="2.181070063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:55:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:55:15.129227699 +0000 UTC m=+1611.714767962" watchObservedRunningTime="2026-10-31 03:55:15.181070063 +0000 UTC m=+1611.766610325"
Oct 31 03:55:15.963731 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:15.963701    4926 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-6ndmq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 03:55:15.964366 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:15.963746    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq" podUID="7d0301a0-5d7b-4305-b4dd-24539a745220" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 03:55:18.963077 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:18.963039    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 03:55:24.391882 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.391847    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"]
Oct 31 03:55:24.400752 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.400735    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.522601 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.522568    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"]
Oct 31 03:55:24.589722 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.589692    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.589880 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.589754    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.589921 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.589902    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.589966 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.589944    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.590010 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.589962    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72wh8\" (UniqueName: \"kubernetes.io/projected/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-kube-api-access-72wh8\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.590086 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.590065    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.590120 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.590105    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.590172 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.590160    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.590241 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.590231    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.691903 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.691802    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.691903 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.691860    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.691903 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.691891    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.692210 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.691972    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.692210 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.692065    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.692210 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.692133    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.692429 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.692224    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-credential-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.692429 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.692262    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.692429 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.692330    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.692429 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.692354    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72wh8\" (UniqueName: \"kubernetes.io/projected/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-kube-api-access-72wh8\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.692429 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.692403    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-workload-socket\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.692673 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.692614    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-workload-certs\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.692673 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.692630    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-data\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.693077 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.693053    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istiod-ca-cert\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.694393 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.694365    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-envoy\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.694762 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.694746    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-podinfo\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.761507 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.761475    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-token\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:24.761642 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:24.761557    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72wh8\" (UniqueName: \"kubernetes.io/projected/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-kube-api-access-72wh8\") pod \"e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") " pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:25.015300 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:25.015248    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:25.441891 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:55:25.441861    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc4ddab3_11f4_42d1_818a_8e1cf7fd34e3.slice/crio-b20455195a21b4c052c30eab49362146f11e9ddf2bc02432807ada5a1eae5315 WatchSource:0}: Error finding container b20455195a21b4c052c30eab49362146f11e9ddf2bc02432807ada5a1eae5315: Status 404 returned error can't find the container with id b20455195a21b4c052c30eab49362146f11e9ddf2bc02432807ada5a1eae5315
Oct 31 03:55:25.445013 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:25.444981    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:55:25.445103 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:25.445048    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:55:25.445103 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:25.445080    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:55:25.445713 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:25.445689    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"]
Oct 31 03:55:26.026056 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:26.026021    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6" event={"ID":"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3","Type":"ContainerStarted","Data":"b035659f10ed9110fc2c2b692f4c176aa9c414be984805ed15d93e352ab5b90b"}
Oct 31 03:55:26.026248 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:26.026078    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6" event={"ID":"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3","Type":"ContainerStarted","Data":"b20455195a21b4c052c30eab49362146f11e9ddf2bc02432807ada5a1eae5315"}
Oct 31 03:55:26.200681 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:26.200621    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6" podStartSLOduration=2.200607989 podStartE2EDuration="2.200607989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:55:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:55:26.200052803 +0000 UTC m=+1622.785593078" watchObservedRunningTime="2026-10-31 03:55:26.200607989 +0000 UTC m=+1622.786148251"
Oct 31 03:55:27.039294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:27.030672    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:27.041844 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:27.041798    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 31 03:55:27.041999 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:27.041873    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6" podUID="bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 31 03:55:28.026293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:28.017728    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 31 03:55:28.026293 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:28.017808    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6" podUID="bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 31 03:55:29.033296 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:29.030351    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 31 03:55:29.033296 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:29.030409    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6" podUID="bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 31 03:55:30.027663 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:30.027625    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 31 03:55:30.027951 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:30.027923    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6" podUID="bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 31 03:55:31.026712 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:31.026669    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 31 03:55:31.027362 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:31.027325    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6" podUID="bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 31 03:55:32.016102 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:32.016063    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 31 03:55:32.016297 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:32.016147    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6" podUID="bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 31 03:55:33.438450 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:33.436678    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:33.438450 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:33.438290    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:33.443511 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:33.443440    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 03:55:35.320683 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:55:35.320644    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:55:54.304997 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.304960    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"]
Oct 31 03:55:54.309286 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.309250    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.343557 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.343523    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.343764 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.343672    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e112a1-fb10-4386-9700-939a8c0892da-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.343764 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.343735    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.343860 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.343844    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.343895 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.343882    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.343930 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.343899    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs2cn\" (UniqueName: \"kubernetes.io/projected/a9e112a1-fb10-4386-9700-939a8c0892da-kube-api-access-vs2cn\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.344149 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.344123    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.344184 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.344163    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e112a1-fb10-4386-9700-939a8c0892da-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.344282 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.344239    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e112a1-fb10-4386-9700-939a8c0892da-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.363298 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.363246    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wk9x4\""
Oct 31 03:55:54.419114 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.419084    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"]
Oct 31 03:55:54.445497 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.445458    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.445497 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.445492    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs2cn\" (UniqueName: \"kubernetes.io/projected/a9e112a1-fb10-4386-9700-939a8c0892da-kube-api-access-vs2cn\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.445732 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.445592    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.445732 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.445614    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e112a1-fb10-4386-9700-939a8c0892da-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.445732 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.445645    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e112a1-fb10-4386-9700-939a8c0892da-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.445732 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.445711    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.445937 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.445816    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e112a1-fb10-4386-9700-939a8c0892da-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.445937 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.445904    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.446049 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.445942    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.446049 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.445976    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.446335 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.446306    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.446581 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.446561    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.446671 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.446637    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e112a1-fb10-4386-9700-939a8c0892da-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.446713 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.446695    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.449226 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.449201    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e112a1-fb10-4386-9700-939a8c0892da-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.450208 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.450181    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.508233 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.508201    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e112a1-fb10-4386-9700-939a8c0892da-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.508510 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.508488    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs2cn\" (UniqueName: \"kubernetes.io/projected/a9e112a1-fb10-4386-9700-939a8c0892da-kube-api-access-vs2cn\") pod \"maas-default-gateway-openshift-default-7854949c9-kbw7q\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:54.628298 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:54.628199    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:55.054893 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:55.054862    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"]
Oct 31 03:55:55.055669 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:55:55.055645    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9e112a1_fb10_4386_9700_939a8c0892da.slice/crio-d83b766588c38a9166f418c0da94ebd6f6c61225efbc510f69cb0d33c826cfab WatchSource:0}: Error finding container d83b766588c38a9166f418c0da94ebd6f6c61225efbc510f69cb0d33c826cfab: Status 404 returned error can't find the container with id d83b766588c38a9166f418c0da94ebd6f6c61225efbc510f69cb0d33c826cfab
Oct 31 03:55:55.058342 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:55.058307    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:55:55.058460 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:55.058390    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:55:55.058460 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:55.058431    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:55:55.294035 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:55.293995    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q" event={"ID":"a9e112a1-fb10-4386-9700-939a8c0892da","Type":"ContainerStarted","Data":"41d5c1e4a7ce9706e185da695eaef7964f1a48d4f877fb846d84fa6b05f65f50"}
Oct 31 03:55:55.294229 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:55.294058    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q" event={"ID":"a9e112a1-fb10-4386-9700-939a8c0892da","Type":"ContainerStarted","Data":"d83b766588c38a9166f418c0da94ebd6f6c61225efbc510f69cb0d33c826cfab"}
Oct 31 03:55:55.420101 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:55.419975    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q" podStartSLOduration=1.41995512 podStartE2EDuration="1.41995512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:55:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:55:55.416363541 +0000 UTC m=+1652.001903803" watchObservedRunningTime="2026-10-31 03:55:55.41995512 +0000 UTC m=+1652.005495382"
Oct 31 03:55:55.629057 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:55.629021    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:55:56.634294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:56.633395    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kbw7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 31 03:55:56.634294 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:56.633461    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q" podUID="a9e112a1-fb10-4386-9700-939a8c0892da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 31 03:55:57.633454 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:57.633400    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kbw7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 31 03:55:57.633714 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:57.633465    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q" podUID="a9e112a1-fb10-4386-9700-939a8c0892da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 31 03:55:58.629291 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:58.628654    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kbw7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 31 03:55:58.629291 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:58.628712    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q" podUID="a9e112a1-fb10-4386-9700-939a8c0892da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 31 03:55:59.628744 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:59.628707    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kbw7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 31 03:55:59.629052 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:55:59.629021    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q" podUID="a9e112a1-fb10-4386-9700-939a8c0892da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 31 03:56:00.633776 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:00.633738    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kbw7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 31 03:56:00.634171 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:00.633800    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q" podUID="a9e112a1-fb10-4386-9700-939a8c0892da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 31 03:56:01.629489 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:01.629452    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kbw7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 31 03:56:01.629660 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:01.629518    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q" podUID="a9e112a1-fb10-4386-9700-939a8c0892da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 31 03:56:02.628838 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:02.628755    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kbw7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Oct 31 03:56:02.628838 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:02.628812    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q" podUID="a9e112a1-fb10-4386-9700-939a8c0892da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Oct 31 03:56:04.185994 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:04.185963    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:56:04.186585 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:04.186431    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:56:04.187514 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:04.187493    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 03:56:05.322027 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:56:05.321957    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:56:09.417540 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.417503    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"]
Oct 31 03:56:09.423002 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.422977    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.493465 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.493426    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.493620 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.493475    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r84zg\" (UniqueName: \"kubernetes.io/projected/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-kube-api-access-r84zg\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.493620 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.493589    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.493723 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.493641    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.493807 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.493785    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.493852 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.493830    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.493935 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.493915    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.493990 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.493975    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.494041 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.494028    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.543594 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.543559    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"]
Oct 31 03:56:09.595184 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.595144    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.595184 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.595182    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.595459 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.595230    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.595459 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.595310    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.595459 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.595335    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r84zg\" (UniqueName: \"kubernetes.io/projected/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-kube-api-access-r84zg\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.595459 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.595383    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.595459 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.595432    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.595714 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.595494    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.595714 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.595604    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.596012 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.595990    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.596289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.596233    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.596359 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.596297    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.596419 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.596376    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.596477 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.596428    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.598058 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.598033    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.598399 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.598378    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.657992 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.657957    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.658164 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.658134    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r84zg\" (UniqueName: \"kubernetes.io/projected/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-kube-api-access-r84zg\") pod \"maas-default-gateway-openshift-default-7854949c9-mbxvj\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:09.737150 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:09.737123    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:10.157736 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:56:10.157702    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9e8b83e_ccbd_4f1f_86a3_4f0cf2025515.slice/crio-a7945332f3cbf2e0f783f17b1a39f3e6fceccc3dae73d3261f2b1045b4a4cdb9 WatchSource:0}: Error finding container a7945332f3cbf2e0f783f17b1a39f3e6fceccc3dae73d3261f2b1045b4a4cdb9: Status 404 returned error can't find the container with id a7945332f3cbf2e0f783f17b1a39f3e6fceccc3dae73d3261f2b1045b4a4cdb9
Oct 31 03:56:10.160489 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:10.160454    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:56:10.160600 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:10.160522    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:56:10.160600 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:10.160552    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:56:10.161976 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:10.161953    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"]
Oct 31 03:56:10.415842 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:10.415737    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" event={"ID":"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515","Type":"ContainerStarted","Data":"39ef6d55bd02c6c373302661d5be7069e79f72c5782008a859484044c80a61be"}
Oct 31 03:56:10.415842 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:10.415811    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" event={"ID":"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515","Type":"ContainerStarted","Data":"a7945332f3cbf2e0f783f17b1a39f3e6fceccc3dae73d3261f2b1045b4a4cdb9"}
Oct 31 03:56:10.539924 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:10.539872    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" podStartSLOduration=1.539857981 podStartE2EDuration="1.539857981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:56:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:56:10.537977964 +0000 UTC m=+1667.123518227" watchObservedRunningTime="2026-10-31 03:56:10.539857981 +0000 UTC m=+1667.125398222"
Oct 31 03:56:10.738142 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:10.738107    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:11.739613 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:11.739574    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mbxvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 31 03:56:11.740012 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:11.739630    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" podUID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 31 03:56:12.738019 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:12.737985    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mbxvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 31 03:56:12.738203 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:12.738037    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" podUID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 31 03:56:13.738520 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:13.738488    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mbxvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 31 03:56:13.738959 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:13.738543    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" podUID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 31 03:56:14.738516 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:14.738473    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mbxvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 31 03:56:14.738708 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:14.738526    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" podUID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 31 03:56:15.738429 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:15.738393    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mbxvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 31 03:56:15.738589 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:15.738448    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" podUID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 31 03:56:16.737770 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:16.737732    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mbxvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 31 03:56:16.738169 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:16.737789    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" podUID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 31 03:56:17.737712 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:17.737675    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mbxvj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 31 03:56:17.737897 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:17.737734    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" podUID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 31 03:56:18.886888 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:18.886842    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:18.887375 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:18.887203    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:18.888051 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:56:18.888029    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 03:56:35.318229 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:56:35.318191    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:57:05.312518 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:57:05.312484    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:57:35.315751 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:57:35.315709    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:57:55.228974 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.228942    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"]
Oct 31 03:57:55.232839 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.232823    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.327346 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.327255    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.327529 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.327388    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/170d646b-12d9-47eb-88db-232869e6854e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.327611 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.327585    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.327684 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.327635    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/170d646b-12d9-47eb-88db-232869e6854e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.327744 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.327694    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.327800 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.327755    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.327845 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.327802    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gt6l\" (UniqueName: \"kubernetes.io/projected/170d646b-12d9-47eb-88db-232869e6854e-kube-api-access-5gt6l\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.327879 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.327861    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.327912 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.327899    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/170d646b-12d9-47eb-88db-232869e6854e-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.343093 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.343068    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"]
Oct 31 03:57:55.429652 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.429612    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/170d646b-12d9-47eb-88db-232869e6854e-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.429847 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.429744    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.429847 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.429772    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/170d646b-12d9-47eb-88db-232869e6854e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.429965 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.429889    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.429965 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.429931    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/170d646b-12d9-47eb-88db-232869e6854e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.430070 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.429992    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.430070 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.430053    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.430170 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.430104    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gt6l\" (UniqueName: \"kubernetes.io/projected/170d646b-12d9-47eb-88db-232869e6854e-kube-api-access-5gt6l\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.430221 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.430163    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.430221 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.430176    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.430359 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.430235    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.430514 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.430484    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.430661 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.430633    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/170d646b-12d9-47eb-88db-232869e6854e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.430724 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.430684    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.432410 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.432386    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.432513 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.432486    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/170d646b-12d9-47eb-88db-232869e6854e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.492471 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.492407    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/170d646b-12d9-47eb-88db-232869e6854e-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.492593 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.492555    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gt6l\" (UniqueName: \"kubernetes.io/projected/170d646b-12d9-47eb-88db-232869e6854e-kube-api-access-5gt6l\") pod \"maas-default-gateway-openshift-default-7854949c9-qk79t\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.545908 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.545879    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:55.967384 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:57:55.967344    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod170d646b_12d9_47eb_88db_232869e6854e.slice/crio-3dbc46a0966c4cf9588743d9b9584d405bbf6ed059f44991b5690cb90dab9c40 WatchSource:0}: Error finding container 3dbc46a0966c4cf9588743d9b9584d405bbf6ed059f44991b5690cb90dab9c40: Status 404 returned error can't find the container with id 3dbc46a0966c4cf9588743d9b9584d405bbf6ed059f44991b5690cb90dab9c40
Oct 31 03:57:55.970322 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.970248    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:57:55.970434 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.970378    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:57:55.970434 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.970423    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:57:55.971317 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:55.971296    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"]
Oct 31 03:57:56.141123 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:56.141088    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t" event={"ID":"170d646b-12d9-47eb-88db-232869e6854e","Type":"ContainerStarted","Data":"10d0d710a4e75b68d6bad90291c2e5253242ae5848a77c865601471bd11988b2"}
Oct 31 03:57:56.141333 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:56.141144    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t" event={"ID":"170d646b-12d9-47eb-88db-232869e6854e","Type":"ContainerStarted","Data":"3dbc46a0966c4cf9588743d9b9584d405bbf6ed059f44991b5690cb90dab9c40"}
Oct 31 03:57:56.266089 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:56.265976    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t" podStartSLOduration=1.265962776 podStartE2EDuration="1.265962776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:57:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:57:56.26479071 +0000 UTC m=+1772.850330972" watchObservedRunningTime="2026-10-31 03:57:56.265962776 +0000 UTC m=+1772.851503064"
Oct 31 03:57:56.546811 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:56.546721    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:57:57.547728 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:57.547698    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-qk79t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 31 03:57:57.548136 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:57.547745    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t" podUID="170d646b-12d9-47eb-88db-232869e6854e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 31 03:57:58.546755 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:58.546723    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-qk79t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 31 03:57:58.546956 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:58.546780    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t" podUID="170d646b-12d9-47eb-88db-232869e6854e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 31 03:57:59.546839 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:59.546802    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-qk79t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 31 03:57:59.547238 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:57:59.546860    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t" podUID="170d646b-12d9-47eb-88db-232869e6854e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 31 03:58:00.546406 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:00.546377    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-qk79t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 31 03:58:00.546604 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:00.546430    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t" podUID="170d646b-12d9-47eb-88db-232869e6854e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 31 03:58:01.546180 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:01.546144    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-qk79t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 31 03:58:01.546691 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:01.546208    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t" podUID="170d646b-12d9-47eb-88db-232869e6854e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 31 03:58:02.546553 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:02.546511    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-qk79t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 31 03:58:02.547078 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:02.546561    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t" podUID="170d646b-12d9-47eb-88db-232869e6854e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 31 03:58:03.546543 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:03.546510    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-qk79t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 31 03:58:03.546756 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:03.546567    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t" podUID="170d646b-12d9-47eb-88db-232869e6854e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 31 03:58:04.552299 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:04.552249    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:58:04.552835 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:04.552811    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:58:04.553455 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:04.553436    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 03:58:05.319470 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:58:05.319436    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:58:23.728805 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:23.728771    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 03:58:23.731924 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:23.731889    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 03:58:35.317281 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:58:35.317234    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:58:47.302914 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.302871    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"]
Oct 31 03:58:47.307929 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.307907    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.366955 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.366888    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"]
Oct 31 03:58:47.410826 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.410794    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-b4a3a5-openshift-default-dockercfg-rcpcm\""
Oct 31 03:58:47.444407 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.444367    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08979c91-eaa0-4650-ace2-eda96f061f1d-istiod-ca-cert\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.444583 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.444485    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9ppn\" (UniqueName: \"kubernetes.io/projected/08979c91-eaa0-4650-ace2-eda96f061f1d-kube-api-access-k9ppn\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.444634 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.444597    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-podinfo\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.444669 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.444642    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-workload-certs\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.444709 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.444692    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-token\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.444811 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.444790    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-credential-socket\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.444895 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.444831    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-workload-socket\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.444953 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.444890    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-envoy\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.444953 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.444925    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-data\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.546731 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.546699    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-workload-socket\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.546901 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.546756    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-envoy\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.546901 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.546779    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-data\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.546901 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.546838    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08979c91-eaa0-4650-ace2-eda96f061f1d-istiod-ca-cert\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.547062 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.546910    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9ppn\" (UniqueName: \"kubernetes.io/projected/08979c91-eaa0-4650-ace2-eda96f061f1d-kube-api-access-k9ppn\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.547062 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.546979    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-podinfo\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.547062 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.547010    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-workload-certs\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.547062 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.547055    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-token\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.547297 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.547154    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-credential-socket\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.547297 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.547211    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-workload-socket\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.547414 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.547240    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-data\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.547534 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.547510    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-credential-socket\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.547673 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.547542    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-workload-certs\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.547751 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.547684    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08979c91-eaa0-4650-ace2-eda96f061f1d-istiod-ca-cert\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.549725 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.549699    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-envoy\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.549980 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.549961    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-podinfo\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.610620 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.610537    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-token\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.610768 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.610648    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9ppn\" (UniqueName: \"kubernetes.io/projected/08979c91-eaa0-4650-ace2-eda96f061f1d-kube-api-access-k9ppn\") pod \"e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") " pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:47.624393 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:47.624361    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:48.074324 ip-10-0-130-19 kubenswrapper[4926]: W1031 03:58:48.074290    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08979c91_eaa0_4650_ace2_eda96f061f1d.slice/crio-d45fc344f2b75ad8ddb4c284b5cacf2da4d9e1fbc967e8dfae961b6cc2066841 WatchSource:0}: Error finding container d45fc344f2b75ad8ddb4c284b5cacf2da4d9e1fbc967e8dfae961b6cc2066841: Status 404 returned error can't find the container with id d45fc344f2b75ad8ddb4c284b5cacf2da4d9e1fbc967e8dfae961b6cc2066841
Oct 31 03:58:48.077435 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:48.077413    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"]
Oct 31 03:58:48.077959 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:48.077921    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:58:48.078023 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:48.078006    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:58:48.078070 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:48.078055    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:58:48.516753 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:48.516709    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2" event={"ID":"08979c91-eaa0-4650-ace2-eda96f061f1d","Type":"ContainerStarted","Data":"64a90b70b5f5a03383070b3d5d437c91fec4f2c452f7b39622fabd9dcebb2d87"}
Oct 31 03:58:48.517232 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:48.516778    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2" event={"ID":"08979c91-eaa0-4650-ace2-eda96f061f1d","Type":"ContainerStarted","Data":"d45fc344f2b75ad8ddb4c284b5cacf2da4d9e1fbc967e8dfae961b6cc2066841"}
Oct 31 03:58:48.625239 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:48.625203    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:48.674532 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:48.674480    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2" podStartSLOduration=1.674465236 podStartE2EDuration="1.674465236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:58:48.666323739 +0000 UTC m=+1825.251863999" watchObservedRunningTime="2026-10-31 03:58:48.674465236 +0000 UTC m=+1825.260005498"
Oct 31 03:58:49.631649 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:49.631615    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:50.531025 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:50.530982    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:50.532018 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:50.532000    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 03:58:51.692513 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.692354    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"]
Oct 31 03:58:51.697907 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.697878    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.789229 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.788118    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-22e20532-openshift-default-dockercfg-7dljr\""
Oct 31 03:58:51.818298 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.813182    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-data\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.818580 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.818553    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-workload-certs\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.818712 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.818698    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istiod-ca-cert\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.818803 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.818792    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-envoy\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.818913 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.818900    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcdqx\" (UniqueName: \"kubernetes.io/projected/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-kube-api-access-xcdqx\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.819125 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.819112    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-workload-socket\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.819227 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.819213    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-podinfo\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.827304 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.823636    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-credential-socket\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.827304 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.823802    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-token\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.921870 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.900334    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"]
Oct 31 03:58:51.932289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.926225    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-token\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.932289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.926336    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-data\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.932289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.926372    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-workload-certs\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.932289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.926406    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istiod-ca-cert\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.932289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.926434    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-envoy\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.932289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.926480    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcdqx\" (UniqueName: \"kubernetes.io/projected/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-kube-api-access-xcdqx\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.932289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.926614    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-workload-socket\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.932289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.926643    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-podinfo\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.932289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.926738    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-credential-socket\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.932289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.927231    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-credential-socket\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.932289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.927839    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-data\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.932289 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.928059    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-workload-certs\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.942785 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.942688    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-workload-socket\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:51.983673 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.983632    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-envoy\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:52.009731 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:51.992441    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istiod-ca-cert\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:52.009731 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:52.007564    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcdqx\" (UniqueName: \"kubernetes.io/projected/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-kube-api-access-xcdqx\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:52.013369 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:52.013339    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-token\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:52.018512 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:52.018483    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-podinfo\") pod \"e2e-mt-22e20532-openshift-default-6b75468d65-flzlq\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") " pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:52.315136 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:52.314856    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:52.762176 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:52.762149    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"]
Oct 31 03:58:52.762529 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:52.762160    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:58:52.762529 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:52.762239    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:58:52.762529 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:52.762326    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 03:58:53.575093 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:53.575055    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq" event={"ID":"b8e4ac44-94a2-480e-a2d9-9d41023b06ea","Type":"ContainerStarted","Data":"15814adf9c0abaad7eb49d179b1675d5a16b95b0eead5a7a5d4217a87ade627c"}
Oct 31 03:58:53.575283 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:53.575119    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq" event={"ID":"b8e4ac44-94a2-480e-a2d9-9d41023b06ea","Type":"ContainerStarted","Data":"e2e33a795f6cee259b996137f8ee0563af33718c0201b5755e8cc854af72659b"}
Oct 31 03:58:53.803139 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:53.803078    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq" podStartSLOduration=2.803060098 podStartE2EDuration="2.803060098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 03:58:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 03:58:53.749486282 +0000 UTC m=+1830.335026543" watchObservedRunningTime="2026-10-31 03:58:53.803060098 +0000 UTC m=+1830.388600367"
Oct 31 03:58:54.315338 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:54.315311    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:54.321841 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:54.321816    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:54.582385 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:54.582309    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:58:54.583371 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:58:54.583350    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 03:59:05.321739 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:59:05.321701    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:59:35.312982 ip-10-0-130-19 kubenswrapper[4926]: E1031 03:59:35.312929    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 03:59:40.670372 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:40.670339    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"]
Oct 31 03:59:40.671060 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:40.670569    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6" podUID="77c6a418-701d-4061-b6db-37cf20c502e4" containerName="istio-proxy" containerID="cri-o://ebcb63fefcafe68d0df8d85efc70cab0c2e03e1546130e1066f4966d3da1e594" gracePeriod=30
Oct 31 03:59:40.723348 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:40.723310    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"]
Oct 31 03:59:40.723621 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:40.723574    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc" podUID="71787759-a0b2-43c1-9939-acfc039ca1eb" containerName="istio-proxy" containerID="cri-o://86fe3d311bfdd7fc2ccdd3b61d5e630e4646bb4998a9e8e720e2f04b1151679e" gracePeriod=30
Oct 31 03:59:45.993981 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:45.993949    4926 generic.go:356] "Generic (PLEG): container finished" podID="71787759-a0b2-43c1-9939-acfc039ca1eb" containerID="86fe3d311bfdd7fc2ccdd3b61d5e630e4646bb4998a9e8e720e2f04b1151679e" exitCode=0
Oct 31 03:59:45.994475 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:45.994022    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc" event={"ID":"71787759-a0b2-43c1-9939-acfc039ca1eb","Type":"ContainerDied","Data":"86fe3d311bfdd7fc2ccdd3b61d5e630e4646bb4998a9e8e720e2f04b1151679e"}
Oct 31 03:59:45.995601 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:45.995579    4926 generic.go:356] "Generic (PLEG): container finished" podID="77c6a418-701d-4061-b6db-37cf20c502e4" containerID="ebcb63fefcafe68d0df8d85efc70cab0c2e03e1546130e1066f4966d3da1e594" exitCode=0
Oct 31 03:59:45.995715 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:45.995636    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6" event={"ID":"77c6a418-701d-4061-b6db-37cf20c502e4","Type":"ContainerDied","Data":"ebcb63fefcafe68d0df8d85efc70cab0c2e03e1546130e1066f4966d3da1e594"}
Oct 31 03:59:45.995715 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:45.995670    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6" event={"ID":"77c6a418-701d-4061-b6db-37cf20c502e4","Type":"ContainerDied","Data":"d56a45a755337a16a8d70a18104b99f47af3fe37fdf526ed9bfc2db88c4b334e"}
Oct 31 03:59:45.995715 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:45.995687    4926 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d56a45a755337a16a8d70a18104b99f47af3fe37fdf526ed9bfc2db88c4b334e"
Oct 31 03:59:46.019922 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.019896    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:59:46.038703 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.038681    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:59:46.146833 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.146788    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-workload-socket\") pod \"77c6a418-701d-4061-b6db-37cf20c502e4\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") "
Oct 31 03:59:46.147061 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.146841    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71787759-a0b2-43c1-9939-acfc039ca1eb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71787759-a0b2-43c1-9939-acfc039ca1eb-istiod-ca-cert\") pod \"71787759-a0b2-43c1-9939-acfc039ca1eb\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") "
Oct 31 03:59:46.147061 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.146882    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-workload-certs\") pod \"71787759-a0b2-43c1-9939-acfc039ca1eb\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") "
Oct 31 03:59:46.147202 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.146926    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-workload-certs\") pod \"77c6a418-701d-4061-b6db-37cf20c502e4\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") "
Oct 31 03:59:46.147202 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147189    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-envoy\") pod \"71787759-a0b2-43c1-9939-acfc039ca1eb\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") "
Oct 31 03:59:46.147337 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147220    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-token\" (UniqueName: \"kubernetes.io/projected/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-token\") pod \"71787759-a0b2-43c1-9939-acfc039ca1eb\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") "
Oct 31 03:59:46.147337 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147331    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/77c6a418-701d-4061-b6db-37cf20c502e4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77c6a418-701d-4061-b6db-37cf20c502e4-istiod-ca-cert\") pod \"77c6a418-701d-4061-b6db-37cf20c502e4\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") "
Oct 31 03:59:46.147451 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147356    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77c6a418-701d-4061-b6db-37cf20c502e4-istio-token\" (UniqueName: \"kubernetes.io/projected/77c6a418-701d-4061-b6db-37cf20c502e4-istio-token\") pod \"77c6a418-701d-4061-b6db-37cf20c502e4\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") "
Oct 31 03:59:46.147451 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147378    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77c6a418-701d-4061-b6db-37cf20c502e4-kube-api-access-sxm98\" (UniqueName: \"kubernetes.io/projected/77c6a418-701d-4061-b6db-37cf20c502e4-kube-api-access-sxm98\") pod \"77c6a418-701d-4061-b6db-37cf20c502e4\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") "
Oct 31 03:59:46.147451 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147353    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-workload-certs" pod "71787759-a0b2-43c1-9939-acfc039ca1eb" (UID: "71787759-a0b2-43c1-9939-acfc039ca1eb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:46.147451 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147421    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71787759-a0b2-43c1-9939-acfc039ca1eb-kube-api-access-ts2n4\" (UniqueName: \"kubernetes.io/projected/71787759-a0b2-43c1-9939-acfc039ca1eb-kube-api-access-ts2n4\") pod \"71787759-a0b2-43c1-9939-acfc039ca1eb\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") "
Oct 31 03:59:46.147720 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147476    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-istio-data\") pod \"77c6a418-701d-4061-b6db-37cf20c502e4\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") "
Oct 31 03:59:46.147720 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147571    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-credential-socket\") pod \"71787759-a0b2-43c1-9939-acfc039ca1eb\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") "
Oct 31 03:59:46.147720 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147659    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-istio-envoy\") pod \"77c6a418-701d-4061-b6db-37cf20c502e4\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") "
Oct 31 03:59:46.147720 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147688    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/77c6a418-701d-4061-b6db-37cf20c502e4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77c6a418-701d-4061-b6db-37cf20c502e4-istio-podinfo\") pod \"77c6a418-701d-4061-b6db-37cf20c502e4\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") "
Oct 31 03:59:46.147893 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147750    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77c6a418-701d-4061-b6db-37cf20c502e4-istiod-ca-cert" pod "77c6a418-701d-4061-b6db-37cf20c502e4" (UID: "77c6a418-701d-4061-b6db-37cf20c502e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:59:46.147893 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147768    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-podinfo\") pod \"71787759-a0b2-43c1-9939-acfc039ca1eb\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") "
Oct 31 03:59:46.147893 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147823    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-credential-socket\") pod \"77c6a418-701d-4061-b6db-37cf20c502e4\" (UID: \"77c6a418-701d-4061-b6db-37cf20c502e4\") "
Oct 31 03:59:46.147893 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147867    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-workload-socket\") pod \"71787759-a0b2-43c1-9939-acfc039ca1eb\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") "
Oct 31 03:59:46.148086 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.147925    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-data\") pod \"71787759-a0b2-43c1-9939-acfc039ca1eb\" (UID: \"71787759-a0b2-43c1-9939-acfc039ca1eb\") "
Oct 31 03:59:46.148086 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.148020    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-istio-data" pod "77c6a418-701d-4061-b6db-37cf20c502e4" (UID: "77c6a418-701d-4061-b6db-37cf20c502e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:46.149490 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.148462    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-workload-socket" pod "77c6a418-701d-4061-b6db-37cf20c502e4" (UID: "77c6a418-701d-4061-b6db-37cf20c502e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:46.149490 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.148826    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.149490 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.148852    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.149490 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.148868    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77c6a418-701d-4061-b6db-37cf20c502e4-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.149490 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.148882    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.149490 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.149078    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-credential-socket" pod "77c6a418-701d-4061-b6db-37cf20c502e4" (UID: "77c6a418-701d-4061-b6db-37cf20c502e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:46.152509 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.151604    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-workload-certs" pod "77c6a418-701d-4061-b6db-37cf20c502e4" (UID: "77c6a418-701d-4061-b6db-37cf20c502e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:46.155248 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.155202    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-workload-socket" pod "71787759-a0b2-43c1-9939-acfc039ca1eb" (UID: "71787759-a0b2-43c1-9939-acfc039ca1eb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:46.156122 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.155402    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-data" pod "71787759-a0b2-43c1-9939-acfc039ca1eb" (UID: "71787759-a0b2-43c1-9939-acfc039ca1eb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:46.156122 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.155428    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77c6a418-701d-4061-b6db-37cf20c502e4-istio-token" pod "77c6a418-701d-4061-b6db-37cf20c502e4" (UID: "77c6a418-701d-4061-b6db-37cf20c502e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:59:46.156122 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.155578    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-credential-socket" pod "71787759-a0b2-43c1-9939-acfc039ca1eb" (UID: "71787759-a0b2-43c1-9939-acfc039ca1eb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:46.156122 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.155599    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71787759-a0b2-43c1-9939-acfc039ca1eb-istiod-ca-cert" pod "71787759-a0b2-43c1-9939-acfc039ca1eb" (UID: "71787759-a0b2-43c1-9939-acfc039ca1eb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 03:59:46.156485 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.156460    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77c6a418-701d-4061-b6db-37cf20c502e4-istio-podinfo" pod "77c6a418-701d-4061-b6db-37cf20c502e4" (UID: "77c6a418-701d-4061-b6db-37cf20c502e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 03:59:46.156804 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.156732    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-podinfo" pod "71787759-a0b2-43c1-9939-acfc039ca1eb" (UID: "71787759-a0b2-43c1-9939-acfc039ca1eb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 03:59:46.158190 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.158103    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-token" pod "71787759-a0b2-43c1-9939-acfc039ca1eb" (UID: "71787759-a0b2-43c1-9939-acfc039ca1eb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:59:46.158190 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.158170    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77c6a418-701d-4061-b6db-37cf20c502e4-kube-api-access-sxm98" pod "77c6a418-701d-4061-b6db-37cf20c502e4" (UID: "77c6a418-701d-4061-b6db-37cf20c502e4"). InnerVolumeSpecName "kube-api-access-sxm98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:59:46.158190 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.158178    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-istio-envoy" pod "77c6a418-701d-4061-b6db-37cf20c502e4" (UID: "77c6a418-701d-4061-b6db-37cf20c502e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:46.159362 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.159330    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-envoy" pod "71787759-a0b2-43c1-9939-acfc039ca1eb" (UID: "71787759-a0b2-43c1-9939-acfc039ca1eb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 03:59:46.159793 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.159767    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71787759-a0b2-43c1-9939-acfc039ca1eb-kube-api-access-ts2n4" pod "71787759-a0b2-43c1-9939-acfc039ca1eb" (UID: "71787759-a0b2-43c1-9939-acfc039ca1eb"). InnerVolumeSpecName "kube-api-access-ts2n4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 03:59:46.250162 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.250070    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.250162 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.250099    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77c6a418-701d-4061-b6db-37cf20c502e4-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.250162 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.250110    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.250162 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.250119    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.250162 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.250127    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.250162 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.250135    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.250162 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.250165    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71787759-a0b2-43c1-9939-acfc039ca1eb-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.250936 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.250173    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77c6a418-701d-4061-b6db-37cf20c502e4-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.250936 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.250182    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.250936 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.250190    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71787759-a0b2-43c1-9939-acfc039ca1eb-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.250936 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.250198    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77c6a418-701d-4061-b6db-37cf20c502e4-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.250936 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.250208    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxm98\" (UniqueName: \"kubernetes.io/projected/77c6a418-701d-4061-b6db-37cf20c502e4-kube-api-access-sxm98\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.250936 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.250217    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ts2n4\" (UniqueName: \"kubernetes.io/projected/71787759-a0b2-43c1-9939-acfc039ca1eb-kube-api-access-ts2n4\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.250936 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.250225    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71787759-a0b2-43c1-9939-acfc039ca1eb-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 03:59:46.947823 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.947733    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 03:59:46.947823 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:46.947806    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6" podUID="77c6a418-701d-4061-b6db-37cf20c502e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 03:59:47.003131 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:47.003102    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"
Oct 31 03:59:47.003131 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:47.003111    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc" event={"ID":"71787759-a0b2-43c1-9939-acfc039ca1eb","Type":"ContainerDied","Data":"284266cfeca6a629344a6f4e9b66a717bcc04d0c93ac8a7b0fded40b6abccca9"}
Oct 31 03:59:47.003657 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:47.003213    4926 scope.go:122] "RemoveContainer" containerID="86fe3d311bfdd7fc2ccdd3b61d5e630e4646bb4998a9e8e720e2f04b1151679e"
Oct 31 03:59:47.003657 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:47.003437    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"
Oct 31 03:59:47.175784 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:47.175740    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"]
Oct 31 03:59:47.226799 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:47.226761    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-lhsz6"]
Oct 31 03:59:47.294014 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:47.293974    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"]
Oct 31 03:59:47.298249 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:47.298223    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-4lnbc"]
Oct 31 03:59:47.539916 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:47.539841    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71787759-a0b2-43c1-9939-acfc039ca1eb" path="/var/lib/kubelet/pods/71787759-a0b2-43c1-9939-acfc039ca1eb/volumes"
Oct 31 03:59:47.540454 ip-10-0-130-19 kubenswrapper[4926]: I1031 03:59:47.540438    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="77c6a418-701d-4061-b6db-37cf20c502e4" path="/var/lib/kubelet/pods/77c6a418-701d-4061-b6db-37cf20c502e4/volumes"
Oct 31 04:00:00.184523 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.184483    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"]
Oct 31 04:00:00.185362 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.185341    4926 cpu_manager.go:414] "containerMap: removing container" podUID="77c6a418-701d-4061-b6db-37cf20c502e4" containerName="istio-proxy"
Oct 31 04:00:00.185362 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.185363    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="77c6a418-701d-4061-b6db-37cf20c502e4" containerName="istio-proxy"
Oct 31 04:00:00.185525 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.185399    4926 cpu_manager.go:414] "containerMap: removing container" podUID="71787759-a0b2-43c1-9939-acfc039ca1eb" containerName="istio-proxy"
Oct 31 04:00:00.185525 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.185408    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71787759-a0b2-43c1-9939-acfc039ca1eb" containerName="istio-proxy"
Oct 31 04:00:00.185623 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.185543    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="71787759-a0b2-43c1-9939-acfc039ca1eb" containerName="istio-proxy"
Oct 31 04:00:00.185623 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.185565    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="77c6a418-701d-4061-b6db-37cf20c502e4" containerName="istio-proxy"
Oct 31 04:00:00.190530 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.190509    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"]
Oct 31 04:00:00.190666 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.190636    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"
Oct 31 04:00:00.194864 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.194843    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"
Oct 31 04:00:00.207799 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.207777    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"]
Oct 31 04:00:00.211884 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.211866    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"
Oct 31 04:00:00.261574 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.261540    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pngvk\" (UniqueName: \"kubernetes.io/projected/4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf-kube-api-access-pngvk\") pod \"maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w\" (UID: \"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"
Oct 31 04:00:00.263747 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.261864    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znm8g\" (UniqueName: \"kubernetes.io/projected/9dec67dd-f744-4a84-8c94-10e91694a5d5-kube-api-access-znm8g\") pod \"maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj\" (UID: \"9dec67dd-f744-4a84-8c94-10e91694a5d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"
Oct 31 04:00:00.263747 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.261910    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/766c0067-9aba-4388-906d-171ff0dd333c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz\" (UID: \"766c0067-9aba-4388-906d-171ff0dd333c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"
Oct 31 04:00:00.263747 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.262229    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dec67dd-f744-4a84-8c94-10e91694a5d5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj\" (UID: \"9dec67dd-f744-4a84-8c94-10e91694a5d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"
Oct 31 04:00:00.263747 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.263009    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"]
Oct 31 04:00:00.263747 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.263249    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cklzq\" (UniqueName: \"kubernetes.io/projected/766c0067-9aba-4388-906d-171ff0dd333c-kube-api-access-cklzq\") pod \"maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz\" (UID: \"766c0067-9aba-4388-906d-171ff0dd333c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"
Oct 31 04:00:00.263747 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.263402    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w\" (UID: \"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"
Oct 31 04:00:00.269829 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.269804    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"]
Oct 31 04:00:00.307053 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.305335    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"]
Oct 31 04:00:00.324703 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.324678    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-78tsm\""
Oct 31 04:00:00.364843 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.364804    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dec67dd-f744-4a84-8c94-10e91694a5d5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj\" (UID: \"9dec67dd-f744-4a84-8c94-10e91694a5d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"
Oct 31 04:00:00.365032 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.364955    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cklzq\" (UniqueName: \"kubernetes.io/projected/766c0067-9aba-4388-906d-171ff0dd333c-kube-api-access-cklzq\") pod \"maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz\" (UID: \"766c0067-9aba-4388-906d-171ff0dd333c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"
Oct 31 04:00:00.365110 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.365088    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w\" (UID: \"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"
Oct 31 04:00:00.365166 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.365129    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pngvk\" (UniqueName: \"kubernetes.io/projected/4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf-kube-api-access-pngvk\") pod \"maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w\" (UID: \"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"
Oct 31 04:00:00.365221 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.365168    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znm8g\" (UniqueName: \"kubernetes.io/projected/9dec67dd-f744-4a84-8c94-10e91694a5d5-kube-api-access-znm8g\") pod \"maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj\" (UID: \"9dec67dd-f744-4a84-8c94-10e91694a5d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"
Oct 31 04:00:00.365221 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.365194    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/766c0067-9aba-4388-906d-171ff0dd333c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz\" (UID: \"766c0067-9aba-4388-906d-171ff0dd333c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"
Oct 31 04:00:00.365700 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.365663    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dec67dd-f744-4a84-8c94-10e91694a5d5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj\" (UID: \"9dec67dd-f744-4a84-8c94-10e91694a5d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"
Oct 31 04:00:00.365857 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.365840    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w\" (UID: \"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"
Oct 31 04:00:00.366169 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.366148    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/766c0067-9aba-4388-906d-171ff0dd333c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz\" (UID: \"766c0067-9aba-4388-906d-171ff0dd333c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"
Oct 31 04:00:00.432343 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.432307    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pngvk\" (UniqueName: \"kubernetes.io/projected/4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf-kube-api-access-pngvk\") pod \"maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w\" (UID: \"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"
Oct 31 04:00:00.432528 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.432404    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cklzq\" (UniqueName: \"kubernetes.io/projected/766c0067-9aba-4388-906d-171ff0dd333c-kube-api-access-cklzq\") pod \"maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz\" (UID: \"766c0067-9aba-4388-906d-171ff0dd333c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"
Oct 31 04:00:00.432528 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.432505    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znm8g\" (UniqueName: \"kubernetes.io/projected/9dec67dd-f744-4a84-8c94-10e91694a5d5-kube-api-access-znm8g\") pod \"maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj\" (UID: \"9dec67dd-f744-4a84-8c94-10e91694a5d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"
Oct 31 04:00:00.507976 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.507940    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"
Oct 31 04:00:00.514456 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.514435    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"
Oct 31 04:00:00.524664 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.524638    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"
Oct 31 04:00:00.893246 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:00:00.893161    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod766c0067_9aba_4388_906d_171ff0dd333c.slice/crio-dffba1ca336a062e65f0b79af538ccff989ed1a2123de5513f99f3bd15a7092f WatchSource:0}: Error finding container dffba1ca336a062e65f0b79af538ccff989ed1a2123de5513f99f3bd15a7092f: Status 404 returned error can't find the container with id dffba1ca336a062e65f0b79af538ccff989ed1a2123de5513f99f3bd15a7092f
Oct 31 04:00:00.895338 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.895320    4926 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 04:00:00.897177 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.897155    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"]
Oct 31 04:00:00.939454 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:00:00.939417    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dec67dd_f744_4a84_8c94_10e91694a5d5.slice/crio-46f1d8d799ecff76de2e2e2b17ce32257546700e5b219b0c4fc4783532ecb767 WatchSource:0}: Error finding container 46f1d8d799ecff76de2e2e2b17ce32257546700e5b219b0c4fc4783532ecb767: Status 404 returned error can't find the container with id 46f1d8d799ecff76de2e2e2b17ce32257546700e5b219b0c4fc4783532ecb767
Oct 31 04:00:00.941826 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:00:00.941801    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bbdcaf9_4e87_408d_8ea2_8a4be2beefcf.slice/crio-8f343d0294f040262d2c0c3177fb25bc2423649af47cd768d862fe85e5dff3ae WatchSource:0}: Error finding container 8f343d0294f040262d2c0c3177fb25bc2423649af47cd768d862fe85e5dff3ae: Status 404 returned error can't find the container with id 8f343d0294f040262d2c0c3177fb25bc2423649af47cd768d862fe85e5dff3ae
Oct 31 04:00:00.943085 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.943069    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"]
Oct 31 04:00:00.946663 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:00.946641    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"]
Oct 31 04:00:01.094878 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:01.094840    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w" event={"ID":"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf","Type":"ContainerStarted","Data":"8f343d0294f040262d2c0c3177fb25bc2423649af47cd768d862fe85e5dff3ae"}
Oct 31 04:00:01.096201 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:01.096178    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj" event={"ID":"9dec67dd-f744-4a84-8c94-10e91694a5d5","Type":"ContainerStarted","Data":"46f1d8d799ecff76de2e2e2b17ce32257546700e5b219b0c4fc4783532ecb767"}
Oct 31 04:00:01.097498 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:01.097472    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz" event={"ID":"766c0067-9aba-4388-906d-171ff0dd333c","Type":"ContainerStarted","Data":"dffba1ca336a062e65f0b79af538ccff989ed1a2123de5513f99f3bd15a7092f"}
Oct 31 04:00:05.323734 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:00:05.323689    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:00:07.143293 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:07.143199    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w" event={"ID":"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf","Type":"ContainerStarted","Data":"3a67407b0d152b0c993829edf7d36ddb9c4608c1db67c131305422f37c322e31"}
Oct 31 04:00:07.152300 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:07.148089    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj" event={"ID":"9dec67dd-f744-4a84-8c94-10e91694a5d5","Type":"ContainerStarted","Data":"be800d0788b54f4204d152bf930ff3ab6848515689ad28e6f2b540a4a1cf0e9d"}
Oct 31 04:00:07.152300 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:07.151399    4926 generic.go:356] "Generic (PLEG): container finished" podID="766c0067-9aba-4388-906d-171ff0dd333c" containerID="0d659235834c41f7af8556324d7e0222e2be2f051357b5d9d7c789b743601459" exitCode=60
Oct 31 04:00:07.152300 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:07.151447    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz" event={"ID":"766c0067-9aba-4388-906d-171ff0dd333c","Type":"ContainerDied","Data":"0d659235834c41f7af8556324d7e0222e2be2f051357b5d9d7c789b743601459"}
Oct 31 04:00:07.152300 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:07.151965    4926 scope.go:122] "RemoveContainer" containerID="0d659235834c41f7af8556324d7e0222e2be2f051357b5d9d7c789b743601459"
Oct 31 04:00:07.264704 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:07.264652    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w" podStartSLOduration=1.562381083 podStartE2EDuration="7.264636521s" totalImagesPullingTime="5.702255438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:00:07.261831819 +0000 UTC m=+1903.847372082" watchObservedRunningTime="2026-10-31 04:00:07.264636521 +0000 UTC m=+1903.850176776"
Oct 31 04:00:07.383221 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:07.383114    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj" podStartSLOduration=1.863235168 podStartE2EDuration="7.383100838s" totalImagesPullingTime="5.51986567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:00:07.380047192 +0000 UTC m=+1903.965587456" watchObservedRunningTime="2026-10-31 04:00:07.383100838 +0000 UTC m=+1903.968641100"
Oct 31 04:00:08.159328 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:08.159261    4926 generic.go:356] "Generic (PLEG): container finished" podID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf" containerID="3a67407b0d152b0c993829edf7d36ddb9c4608c1db67c131305422f37c322e31" exitCode=60
Oct 31 04:00:08.159768 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:08.159364    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w" event={"ID":"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf","Type":"ContainerDied","Data":"3a67407b0d152b0c993829edf7d36ddb9c4608c1db67c131305422f37c322e31"}
Oct 31 04:00:08.159768 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:08.159714    4926 scope.go:122] "RemoveContainer" containerID="3a67407b0d152b0c993829edf7d36ddb9c4608c1db67c131305422f37c322e31"
Oct 31 04:00:08.161154 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:08.161129    4926 generic.go:356] "Generic (PLEG): container finished" podID="9dec67dd-f744-4a84-8c94-10e91694a5d5" containerID="be800d0788b54f4204d152bf930ff3ab6848515689ad28e6f2b540a4a1cf0e9d" exitCode=60
Oct 31 04:00:08.161262 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:08.161216    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj" event={"ID":"9dec67dd-f744-4a84-8c94-10e91694a5d5","Type":"ContainerDied","Data":"be800d0788b54f4204d152bf930ff3ab6848515689ad28e6f2b540a4a1cf0e9d"}
Oct 31 04:00:08.161524 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:08.161505    4926 scope.go:122] "RemoveContainer" containerID="be800d0788b54f4204d152bf930ff3ab6848515689ad28e6f2b540a4a1cf0e9d"
Oct 31 04:00:08.164352 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:08.164333    4926 generic.go:356] "Generic (PLEG): container finished" podID="766c0067-9aba-4388-906d-171ff0dd333c" containerID="336ee6845bc18efafc254046e8efab6c9a280a56d862b0c1e7af9087a1c274ae" exitCode=60
Oct 31 04:00:08.164438 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:08.164388    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz" event={"ID":"766c0067-9aba-4388-906d-171ff0dd333c","Type":"ContainerDied","Data":"336ee6845bc18efafc254046e8efab6c9a280a56d862b0c1e7af9087a1c274ae"}
Oct 31 04:00:08.164541 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:08.164505    4926 scope.go:122] "RemoveContainer" containerID="0d659235834c41f7af8556324d7e0222e2be2f051357b5d9d7c789b743601459"
Oct 31 04:00:08.164816 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:08.164736    4926 scope.go:122] "RemoveContainer" containerID="336ee6845bc18efafc254046e8efab6c9a280a56d862b0c1e7af9087a1c274ae"
Oct 31 04:00:08.164989 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:00:08.164969    4926 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz_odh-ai-gateway-infra(766c0067-9aba-4388-906d-171ff0dd333c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz" podUID="766c0067-9aba-4388-906d-171ff0dd333c"
Oct 31 04:00:09.174204 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:09.174116    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w" event={"ID":"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf","Type":"ContainerStarted","Data":"fc5172a5f74210febc838ac5e7dd15151e08b173596a4ab7742d27bb72d5814b"}
Oct 31 04:00:09.176151 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:09.176127    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj" event={"ID":"9dec67dd-f744-4a84-8c94-10e91694a5d5","Type":"ContainerStarted","Data":"e684f86be1e7d1f84256b468d0d9fb9d3cb950df4e5436b23de051b82860d63b"}
Oct 31 04:00:09.177877 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:09.177860    4926 scope.go:122] "RemoveContainer" containerID="336ee6845bc18efafc254046e8efab6c9a280a56d862b0c1e7af9087a1c274ae"
Oct 31 04:00:09.178128 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:00:09.178110    4926 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz_odh-ai-gateway-infra(766c0067-9aba-4388-906d-171ff0dd333c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz" podUID="766c0067-9aba-4388-906d-171ff0dd333c"
Oct 31 04:00:10.186159 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:10.186124    4926 generic.go:356] "Generic (PLEG): container finished" podID="9dec67dd-f744-4a84-8c94-10e91694a5d5" containerID="e684f86be1e7d1f84256b468d0d9fb9d3cb950df4e5436b23de051b82860d63b" exitCode=60
Oct 31 04:00:10.186654 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:10.186210    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj" event={"ID":"9dec67dd-f744-4a84-8c94-10e91694a5d5","Type":"ContainerDied","Data":"e684f86be1e7d1f84256b468d0d9fb9d3cb950df4e5436b23de051b82860d63b"}
Oct 31 04:00:10.186654 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:10.186360    4926 scope.go:122] "RemoveContainer" containerID="be800d0788b54f4204d152bf930ff3ab6848515689ad28e6f2b540a4a1cf0e9d"
Oct 31 04:00:10.186805 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:10.186675    4926 scope.go:122] "RemoveContainer" containerID="e684f86be1e7d1f84256b468d0d9fb9d3cb950df4e5436b23de051b82860d63b"
Oct 31 04:00:10.187112 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:00:10.187074    4926 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj_odh-ai-gateway-infra(9dec67dd-f744-4a84-8c94-10e91694a5d5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj" podUID="9dec67dd-f744-4a84-8c94-10e91694a5d5"
Oct 31 04:00:10.188536 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:10.188515    4926 generic.go:356] "Generic (PLEG): container finished" podID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf" containerID="fc5172a5f74210febc838ac5e7dd15151e08b173596a4ab7742d27bb72d5814b" exitCode=60
Oct 31 04:00:10.188632 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:10.188547    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w" event={"ID":"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf","Type":"ContainerDied","Data":"fc5172a5f74210febc838ac5e7dd15151e08b173596a4ab7742d27bb72d5814b"}
Oct 31 04:00:10.188886 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:10.188871    4926 scope.go:122] "RemoveContainer" containerID="fc5172a5f74210febc838ac5e7dd15151e08b173596a4ab7742d27bb72d5814b"
Oct 31 04:00:10.189216 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:00:10.189197    4926 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w_odh-ai-gateway-infra(4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w" podUID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf"
Oct 31 04:00:10.199629 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:10.199602    4926 scope.go:122] "RemoveContainer" containerID="3a67407b0d152b0c993829edf7d36ddb9c4608c1db67c131305422f37c322e31"
Oct 31 04:00:11.196736 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:11.196707    4926 scope.go:122] "RemoveContainer" containerID="e684f86be1e7d1f84256b468d0d9fb9d3cb950df4e5436b23de051b82860d63b"
Oct 31 04:00:11.197236 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:00:11.197118    4926 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj_odh-ai-gateway-infra(9dec67dd-f744-4a84-8c94-10e91694a5d5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj" podUID="9dec67dd-f744-4a84-8c94-10e91694a5d5"
Oct 31 04:00:11.198526 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:11.198496    4926 scope.go:122] "RemoveContainer" containerID="fc5172a5f74210febc838ac5e7dd15151e08b173596a4ab7742d27bb72d5814b"
Oct 31 04:00:11.198818 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:00:11.198800    4926 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w_odh-ai-gateway-infra(4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w" podUID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf"
Oct 31 04:00:20.531774 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:20.531741    4926 scope.go:122] "RemoveContainer" containerID="336ee6845bc18efafc254046e8efab6c9a280a56d862b0c1e7af9087a1c274ae"
Oct 31 04:00:20.533681 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:20.533651    4926 scope.go:122] "RemoveContainer" containerID="e684f86be1e7d1f84256b468d0d9fb9d3cb950df4e5436b23de051b82860d63b"
Oct 31 04:00:21.296374 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:21.296331    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj" event={"ID":"9dec67dd-f744-4a84-8c94-10e91694a5d5","Type":"ContainerStarted","Data":"760b820c2403bdcca21eb1e6bb868a0780c646c4286fb34460865d464fa2f90e"}
Oct 31 04:00:21.299907 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:21.299876    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz" event={"ID":"766c0067-9aba-4388-906d-171ff0dd333c","Type":"ContainerStarted","Data":"8bd1fa7b59ce11e157b12d1c88caccac7e1b80cacea1282267643dc53db0ae2c"}
Oct 31 04:00:22.346645 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:22.346591    4926 generic.go:356] "Generic (PLEG): container finished" podID="766c0067-9aba-4388-906d-171ff0dd333c" containerID="8bd1fa7b59ce11e157b12d1c88caccac7e1b80cacea1282267643dc53db0ae2c" exitCode=60
Oct 31 04:00:22.352294 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:22.348288    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz" event={"ID":"766c0067-9aba-4388-906d-171ff0dd333c","Type":"ContainerDied","Data":"8bd1fa7b59ce11e157b12d1c88caccac7e1b80cacea1282267643dc53db0ae2c"}
Oct 31 04:00:22.352627 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:22.352609    4926 scope.go:122] "RemoveContainer" containerID="336ee6845bc18efafc254046e8efab6c9a280a56d862b0c1e7af9087a1c274ae"
Oct 31 04:00:22.357507 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:22.357484    4926 scope.go:122] "RemoveContainer" containerID="8bd1fa7b59ce11e157b12d1c88caccac7e1b80cacea1282267643dc53db0ae2c"
Oct 31 04:00:22.359811 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:00:22.359781    4926 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz_odh-ai-gateway-infra(766c0067-9aba-4388-906d-171ff0dd333c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz" podUID="766c0067-9aba-4388-906d-171ff0dd333c"
Oct 31 04:00:22.364551 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:22.364496    4926 generic.go:356] "Generic (PLEG): container finished" podID="9dec67dd-f744-4a84-8c94-10e91694a5d5" containerID="760b820c2403bdcca21eb1e6bb868a0780c646c4286fb34460865d464fa2f90e" exitCode=60
Oct 31 04:00:22.364712 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:22.364683    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj" event={"ID":"9dec67dd-f744-4a84-8c94-10e91694a5d5","Type":"ContainerDied","Data":"760b820c2403bdcca21eb1e6bb868a0780c646c4286fb34460865d464fa2f90e"}
Oct 31 04:00:22.365355 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:22.365339    4926 scope.go:122] "RemoveContainer" containerID="760b820c2403bdcca21eb1e6bb868a0780c646c4286fb34460865d464fa2f90e"
Oct 31 04:00:22.365832 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:00:22.365814    4926 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj_odh-ai-gateway-infra(9dec67dd-f744-4a84-8c94-10e91694a5d5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj" podUID="9dec67dd-f744-4a84-8c94-10e91694a5d5"
Oct 31 04:00:22.381435 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:22.381412    4926 scope.go:122] "RemoveContainer" containerID="e684f86be1e7d1f84256b468d0d9fb9d3cb950df4e5436b23de051b82860d63b"
Oct 31 04:00:22.467293 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:22.466188    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz" podStartSLOduration=17.482348745 podStartE2EDuration="22.466165702s" totalImagesPullingTime="4.983816957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:00:21.548198593 +0000 UTC m=+1918.133738856" watchObservedRunningTime="2026-10-31 04:00:22.466165702 +0000 UTC m=+1919.051705966"
Oct 31 04:00:22.467293 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:22.467128    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"]
Oct 31 04:00:22.580593 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:22.580463    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"]
Oct 31 04:00:22.582614 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:22.582574    4926 scope.go:122] "RemoveContainer" containerID="fc5172a5f74210febc838ac5e7dd15151e08b173596a4ab7742d27bb72d5814b"
Oct 31 04:00:23.388429 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:23.388330    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w" event={"ID":"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf","Type":"ContainerStarted","Data":"4784a238c4f26684cab84e9f23de4df466169400d278b44e03b302ed4dd7acd7"}
Oct 31 04:00:23.777331 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:23.771478    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"
Oct 31 04:00:23.777331 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:23.775217    4926 scope.go:122] "RemoveContainer" containerID="ebcb63fefcafe68d0df8d85efc70cab0c2e03e1546130e1066f4966d3da1e594"
Oct 31 04:00:23.879381 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:23.879355    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9dec67dd-f744-4a84-8c94-10e91694a5d5-kube-api-access-znm8g\" (UniqueName: \"kubernetes.io/projected/9dec67dd-f744-4a84-8c94-10e91694a5d5-kube-api-access-znm8g\") pod \"9dec67dd-f744-4a84-8c94-10e91694a5d5\" (UID: \"9dec67dd-f744-4a84-8c94-10e91694a5d5\") "
Oct 31 04:00:23.879716 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:23.879694    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9dec67dd-f744-4a84-8c94-10e91694a5d5-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dec67dd-f744-4a84-8c94-10e91694a5d5-maas-api-service-ca\") pod \"9dec67dd-f744-4a84-8c94-10e91694a5d5\" (UID: \"9dec67dd-f744-4a84-8c94-10e91694a5d5\") "
Oct 31 04:00:23.880496 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:23.880473    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dec67dd-f744-4a84-8c94-10e91694a5d5-maas-api-service-ca" pod "9dec67dd-f744-4a84-8c94-10e91694a5d5" (UID: "9dec67dd-f744-4a84-8c94-10e91694a5d5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:23.881816 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:23.881795    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dec67dd-f744-4a84-8c94-10e91694a5d5-kube-api-access-znm8g" pod "9dec67dd-f744-4a84-8c94-10e91694a5d5" (UID: "9dec67dd-f744-4a84-8c94-10e91694a5d5"). InnerVolumeSpecName "kube-api-access-znm8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:23.908022 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:23.907999    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"
Oct 31 04:00:23.981135 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:23.981096    4926 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9dec67dd-f744-4a84-8c94-10e91694a5d5-maas-api-service-ca\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:23.981135 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:23.981132    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znm8g\" (UniqueName: \"kubernetes.io/projected/9dec67dd-f744-4a84-8c94-10e91694a5d5-kube-api-access-znm8g\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:24.083045 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.082949    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/766c0067-9aba-4388-906d-171ff0dd333c-kube-api-access-cklzq\" (UniqueName: \"kubernetes.io/projected/766c0067-9aba-4388-906d-171ff0dd333c-kube-api-access-cklzq\") pod \"766c0067-9aba-4388-906d-171ff0dd333c\" (UID: \"766c0067-9aba-4388-906d-171ff0dd333c\") "
Oct 31 04:00:24.083045 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.083028    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/766c0067-9aba-4388-906d-171ff0dd333c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/766c0067-9aba-4388-906d-171ff0dd333c-maas-api-service-ca\") pod \"766c0067-9aba-4388-906d-171ff0dd333c\" (UID: \"766c0067-9aba-4388-906d-171ff0dd333c\") "
Oct 31 04:00:24.083534 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.083510    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/766c0067-9aba-4388-906d-171ff0dd333c-maas-api-service-ca" pod "766c0067-9aba-4388-906d-171ff0dd333c" (UID: "766c0067-9aba-4388-906d-171ff0dd333c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:24.087327 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.086231    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/766c0067-9aba-4388-906d-171ff0dd333c-kube-api-access-cklzq" pod "766c0067-9aba-4388-906d-171ff0dd333c" (UID: "766c0067-9aba-4388-906d-171ff0dd333c"). InnerVolumeSpecName "kube-api-access-cklzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:24.184926 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.184858    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cklzq\" (UniqueName: \"kubernetes.io/projected/766c0067-9aba-4388-906d-171ff0dd333c-kube-api-access-cklzq\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:24.184926 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.184900    4926 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/766c0067-9aba-4388-906d-171ff0dd333c-maas-api-service-ca\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:24.446349 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.444694    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj" event={"ID":"9dec67dd-f744-4a84-8c94-10e91694a5d5","Type":"ContainerDied","Data":"46f1d8d799ecff76de2e2e2b17ce32257546700e5b219b0c4fc4783532ecb767"}
Oct 31 04:00:24.446349 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.444770    4926 scope.go:122] "RemoveContainer" containerID="760b820c2403bdcca21eb1e6bb868a0780c646c4286fb34460865d464fa2f90e"
Oct 31 04:00:24.446349 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.445066    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"
Oct 31 04:00:24.469301 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.457260    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz" event={"ID":"766c0067-9aba-4388-906d-171ff0dd333c","Type":"ContainerDied","Data":"dffba1ca336a062e65f0b79af538ccff989ed1a2123de5513f99f3bd15a7092f"}
Oct 31 04:00:24.469301 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.457421    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"
Oct 31 04:00:24.532947 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.501986    4926 generic.go:356] "Generic (PLEG): container finished" podID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf" containerID="4784a238c4f26684cab84e9f23de4df466169400d278b44e03b302ed4dd7acd7" exitCode=60
Oct 31 04:00:24.532947 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.502033    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w" event={"ID":"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf","Type":"ContainerDied","Data":"4784a238c4f26684cab84e9f23de4df466169400d278b44e03b302ed4dd7acd7"}
Oct 31 04:00:24.532947 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.502452    4926 scope.go:122] "RemoveContainer" containerID="4784a238c4f26684cab84e9f23de4df466169400d278b44e03b302ed4dd7acd7"
Oct 31 04:00:24.532947 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:00:24.502827    4926 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w_odh-ai-gateway-infra(4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w" podUID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf"
Oct 31 04:00:24.575293 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.567516    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"]
Oct 31 04:00:24.696949 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.696077    4926 scope.go:122] "RemoveContainer" containerID="8bd1fa7b59ce11e157b12d1c88caccac7e1b80cacea1282267643dc53db0ae2c"
Oct 31 04:00:24.906671 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:24.904465    4926 scope.go:122] "RemoveContainer" containerID="fc5172a5f74210febc838ac5e7dd15151e08b173596a4ab7742d27bb72d5814b"
Oct 31 04:00:25.225321 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:25.206555    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"]
Oct 31 04:00:25.248402 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:25.248357    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-412ba5-29890320-lkwrj"]
Oct 31 04:00:25.298578 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:25.297322    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"]
Oct 31 04:00:25.319293 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:25.317970    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ae8e27-29890320-wszxz"]
Oct 31 04:00:25.589860 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:25.589816    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="766c0067-9aba-4388-906d-171ff0dd333c" path="/var/lib/kubelet/pods/766c0067-9aba-4388-906d-171ff0dd333c/volumes"
Oct 31 04:00:25.590424 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:25.590401    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9dec67dd-f744-4a84-8c94-10e91694a5d5" path="/var/lib/kubelet/pods/9dec67dd-f744-4a84-8c94-10e91694a5d5/volumes"
Oct 31 04:00:25.982210 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:25.982172    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"
Oct 31 04:00:26.001062 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:26.001031    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf-kube-api-access-pngvk\" (UniqueName: \"kubernetes.io/projected/4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf-kube-api-access-pngvk\") pod \"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf\" (UID: \"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf\") "
Oct 31 04:00:26.001206 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:26.001115    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf-maas-api-service-ca\") pod \"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf\" (UID: \"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf\") "
Oct 31 04:00:26.001886 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:26.001860    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf-maas-api-service-ca" pod "4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf" (UID: "4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:26.003699 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:26.003678    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf-kube-api-access-pngvk" pod "4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf" (UID: "4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf"). InnerVolumeSpecName "kube-api-access-pngvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:26.106010 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:26.103789    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pngvk\" (UniqueName: \"kubernetes.io/projected/4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf-kube-api-access-pngvk\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:26.106010 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:26.103823    4926 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf-maas-api-service-ca\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:26.603812 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:26.603768    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w" event={"ID":"4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf","Type":"ContainerDied","Data":"8f343d0294f040262d2c0c3177fb25bc2423649af47cd768d862fe85e5dff3ae"}
Oct 31 04:00:26.604218 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:26.603846    4926 scope.go:122] "RemoveContainer" containerID="4784a238c4f26684cab84e9f23de4df466169400d278b44e03b302ed4dd7acd7"
Oct 31 04:00:26.604218 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:26.603983    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"
Oct 31 04:00:26.830738 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:26.830702    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"]
Oct 31 04:00:26.866385 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:26.866295    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-d7b7bc-29890320-kl28w"]
Oct 31 04:00:27.555171 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:27.555140    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf" path="/var/lib/kubelet/pods/4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf/volumes"
Oct 31 04:00:31.655959 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:31.655863    4926 generic.go:356] "Generic (PLEG): container finished" podID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" containerID="15814adf9c0abaad7eb49d179b1675d5a16b95b0eead5a7a5d4217a87ade627c" exitCode=0
Oct 31 04:00:31.656483 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:31.656012    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq" event={"ID":"b8e4ac44-94a2-480e-a2d9-9d41023b06ea","Type":"ContainerDied","Data":"15814adf9c0abaad7eb49d179b1675d5a16b95b0eead5a7a5d4217a87ade627c"}
Oct 31 04:00:31.656837 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:31.656656    4926 scope.go:122] "RemoveContainer" containerID="15814adf9c0abaad7eb49d179b1675d5a16b95b0eead5a7a5d4217a87ade627c"
Oct 31 04:00:31.658986 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:31.658958    4926 generic.go:356] "Generic (PLEG): container finished" podID="08979c91-eaa0-4650-ace2-eda96f061f1d" containerID="64a90b70b5f5a03383070b3d5d437c91fec4f2c452f7b39622fabd9dcebb2d87" exitCode=0
Oct 31 04:00:31.659095 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:31.659028    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2" event={"ID":"08979c91-eaa0-4650-ace2-eda96f061f1d","Type":"ContainerDied","Data":"64a90b70b5f5a03383070b3d5d437c91fec4f2c452f7b39622fabd9dcebb2d87"}
Oct 31 04:00:31.659499 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:31.659455    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 04:00:31.659578 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:31.659542    4926 scope.go:122] "RemoveContainer" containerID="64a90b70b5f5a03383070b3d5d437c91fec4f2c452f7b39622fabd9dcebb2d87"
Oct 31 04:00:31.659634 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:31.659609    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 04:00:31.659738 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:31.659716    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 04:00:31.661577 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:31.661546    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 04:00:31.661693 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:31.661614    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 04:00:31.661693 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:31.661647    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 04:00:32.667617 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:32.667530    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq" event={"ID":"b8e4ac44-94a2-480e-a2d9-9d41023b06ea","Type":"ContainerStarted","Data":"b54c8c42696db542485cff9251d4148060b18c0a541a2e8be8d08d1e477a1184"}
Oct 31 04:00:32.669571 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:32.669548    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2" event={"ID":"08979c91-eaa0-4650-ace2-eda96f061f1d","Type":"ContainerStarted","Data":"d689544d444ee0b479f8ab9b1a885a4b7516094cdeaae9c1fc2a457fa3a04c88"}
Oct 31 04:00:33.329298 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:33.328658    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 04:00:33.329298 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:33.328816    4926 patch_prober.go:29] interesting pod/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 31 04:00:33.329298 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:33.328861    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq" podUID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 31 04:00:33.643215 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:33.643116    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 04:00:33.658624 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:33.658577    4926 patch_prober.go:29] interesting pod/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 31 04:00:33.658911 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:33.658880    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2" podUID="08979c91-eaa0-4650-ace2-eda96f061f1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 31 04:00:34.502934 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:34.492861    4926 patch_prober.go:29] interesting pod/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 31 04:00:34.502934 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:34.492921    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq" podUID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 31 04:00:34.633291 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:34.627644    4926 patch_prober.go:29] interesting pod/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 31 04:00:34.633291 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:34.627705    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2" podUID="08979c91-eaa0-4650-ace2-eda96f061f1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 31 04:00:35.333963 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:35.333037    4926 patch_prober.go:29] interesting pod/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 31 04:00:35.333963 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:35.333098    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq" podUID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 31 04:00:35.369133 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:00:35.369061    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:00:35.553204 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:35.551408    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 04:00:35.633044 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:35.632953    4926 patch_prober.go:29] interesting pod/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 31 04:00:35.633322 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:35.633291    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2" podUID="08979c91-eaa0-4650-ace2-eda96f061f1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 31 04:00:36.315684 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:36.315645    4926 patch_prober.go:29] interesting pod/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 31 04:00:36.316016 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:36.315976    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq" podUID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 31 04:00:36.626840 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:36.626751    4926 patch_prober.go:29] interesting pod/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 31 04:00:36.627455 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:36.627423    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2" podUID="08979c91-eaa0-4650-ace2-eda96f061f1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 31 04:00:37.316694 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:37.316653    4926 patch_prober.go:29] interesting pod/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 31 04:00:37.316993 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:37.316961    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq" podUID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 31 04:00:37.632354 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:37.631759    4926 patch_prober.go:29] interesting pod/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 31 04:00:37.632354 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:37.631825    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2" podUID="08979c91-eaa0-4650-ace2-eda96f061f1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 31 04:00:38.316830 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:38.316793    4926 patch_prober.go:29] interesting pod/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Oct 31 04:00:38.317168 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:38.317114    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq" podUID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Oct 31 04:00:38.741451 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:38.741414    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 04:00:38.743599 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:38.743568    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 04:00:39.319988 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:39.319958    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 04:00:39.320401 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:39.320381    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 04:00:39.321118 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:39.321098    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 04:00:42.169442 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:42.169408    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"]
Oct 31 04:00:42.169845 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:42.169603    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm" podUID="136c00fa-e898-4bfd-9b48-9241fba6740e" containerName="istio-proxy" containerID="cri-o://d17a6464f2bdb469ae68d3613c098a3cfd60fe322756b2e7772cc6c3b4841a81" gracePeriod=30
Oct 31 04:00:43.696635 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:43.696603    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:00:43.697028 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:43.696666    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm" podUID="136c00fa-e898-4bfd-9b48-9241fba6740e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:00:47.443245 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.443178    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 04:00:47.484925 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.484888    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-podinfo\") pod \"136c00fa-e898-4bfd-9b48-9241fba6740e\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") "
Oct 31 04:00:47.485100 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.485000    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-credential-socket\") pod \"136c00fa-e898-4bfd-9b48-9241fba6740e\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") "
Oct 31 04:00:47.485100 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.485091    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/136c00fa-e898-4bfd-9b48-9241fba6740e-kube-api-access-cpbx6\" (UniqueName: \"kubernetes.io/projected/136c00fa-e898-4bfd-9b48-9241fba6740e-kube-api-access-cpbx6\") pod \"136c00fa-e898-4bfd-9b48-9241fba6740e\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") "
Oct 31 04:00:47.485320 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.485216    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-token\" (UniqueName: \"kubernetes.io/projected/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-token\") pod \"136c00fa-e898-4bfd-9b48-9241fba6740e\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") "
Oct 31 04:00:47.485423 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.485317    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-credential-socket" pod "136c00fa-e898-4bfd-9b48-9241fba6740e" (UID: "136c00fa-e898-4bfd-9b48-9241fba6740e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:47.485423 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.485369    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-data\") pod \"136c00fa-e898-4bfd-9b48-9241fba6740e\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") "
Oct 31 04:00:47.485423 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.485400    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-workload-socket\") pod \"136c00fa-e898-4bfd-9b48-9241fba6740e\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") "
Oct 31 04:00:47.485589 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.485520    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-envoy\") pod \"136c00fa-e898-4bfd-9b48-9241fba6740e\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") "
Oct 31 04:00:47.485589 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.485582    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/136c00fa-e898-4bfd-9b48-9241fba6740e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/136c00fa-e898-4bfd-9b48-9241fba6740e-istiod-ca-cert\") pod \"136c00fa-e898-4bfd-9b48-9241fba6740e\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") "
Oct 31 04:00:47.485690 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.485628    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-workload-certs\") pod \"136c00fa-e898-4bfd-9b48-9241fba6740e\" (UID: \"136c00fa-e898-4bfd-9b48-9241fba6740e\") "
Oct 31 04:00:47.486154 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.486126    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-workload-socket" pod "136c00fa-e898-4bfd-9b48-9241fba6740e" (UID: "136c00fa-e898-4bfd-9b48-9241fba6740e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:47.486417 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.486394    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-workload-certs" pod "136c00fa-e898-4bfd-9b48-9241fba6740e" (UID: "136c00fa-e898-4bfd-9b48-9241fba6740e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:47.486496 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.486452    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:47.486496 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.486474    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:47.486691 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.486671    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-data" pod "136c00fa-e898-4bfd-9b48-9241fba6740e" (UID: "136c00fa-e898-4bfd-9b48-9241fba6740e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:47.486993 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.486957    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/136c00fa-e898-4bfd-9b48-9241fba6740e-istiod-ca-cert" pod "136c00fa-e898-4bfd-9b48-9241fba6740e" (UID: "136c00fa-e898-4bfd-9b48-9241fba6740e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:00:47.487787 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.487760    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/136c00fa-e898-4bfd-9b48-9241fba6740e-kube-api-access-cpbx6" pod "136c00fa-e898-4bfd-9b48-9241fba6740e" (UID: "136c00fa-e898-4bfd-9b48-9241fba6740e"). InnerVolumeSpecName "kube-api-access-cpbx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:47.487977 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.487948    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-podinfo" pod "136c00fa-e898-4bfd-9b48-9241fba6740e" (UID: "136c00fa-e898-4bfd-9b48-9241fba6740e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:00:47.487977 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.487960    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-token" pod "136c00fa-e898-4bfd-9b48-9241fba6740e" (UID: "136c00fa-e898-4bfd-9b48-9241fba6740e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:00:47.488630 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.488609    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-envoy" pod "136c00fa-e898-4bfd-9b48-9241fba6740e" (UID: "136c00fa-e898-4bfd-9b48-9241fba6740e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:00:47.588395 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.588360    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:47.588395 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.588395    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:47.588588 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.588413    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:47.588588 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.588429    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/136c00fa-e898-4bfd-9b48-9241fba6740e-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:47.588588 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.588447    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/136c00fa-e898-4bfd-9b48-9241fba6740e-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:47.588588 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.588463    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/136c00fa-e898-4bfd-9b48-9241fba6740e-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:47.588588 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.588478    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpbx6\" (UniqueName: \"kubernetes.io/projected/136c00fa-e898-4bfd-9b48-9241fba6740e-kube-api-access-cpbx6\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:00:47.853300 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.853247    4926 generic.go:356] "Generic (PLEG): container finished" podID="136c00fa-e898-4bfd-9b48-9241fba6740e" containerID="d17a6464f2bdb469ae68d3613c098a3cfd60fe322756b2e7772cc6c3b4841a81" exitCode=0
Oct 31 04:00:47.853449 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.853329    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm" event={"ID":"136c00fa-e898-4bfd-9b48-9241fba6740e","Type":"ContainerDied","Data":"d17a6464f2bdb469ae68d3613c098a3cfd60fe322756b2e7772cc6c3b4841a81"}
Oct 31 04:00:47.853449 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.853381    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm" event={"ID":"136c00fa-e898-4bfd-9b48-9241fba6740e","Type":"ContainerDied","Data":"01c3056c858f18f6d7ffe4630f1df413d6241e941e31de671f71d118cd183a45"}
Oct 31 04:00:47.853449 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.853401    4926 scope.go:122] "RemoveContainer" containerID="d17a6464f2bdb469ae68d3613c098a3cfd60fe322756b2e7772cc6c3b4841a81"
Oct 31 04:00:47.853449 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.853339    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"
Oct 31 04:00:47.863105 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.863082    4926 scope.go:122] "RemoveContainer" containerID="d17a6464f2bdb469ae68d3613c098a3cfd60fe322756b2e7772cc6c3b4841a81"
Oct 31 04:00:47.863368 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:00:47.863349    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d17a6464f2bdb469ae68d3613c098a3cfd60fe322756b2e7772cc6c3b4841a81\": container with ID starting with d17a6464f2bdb469ae68d3613c098a3cfd60fe322756b2e7772cc6c3b4841a81 not found: ID does not exist" containerID="d17a6464f2bdb469ae68d3613c098a3cfd60fe322756b2e7772cc6c3b4841a81"
Oct 31 04:00:47.863443 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:47.863375    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d17a6464f2bdb469ae68d3613c098a3cfd60fe322756b2e7772cc6c3b4841a81"} err="failed to get container status \"d17a6464f2bdb469ae68d3613c098a3cfd60fe322756b2e7772cc6c3b4841a81\": rpc error: code = NotFound desc = could not find container \"d17a6464f2bdb469ae68d3613c098a3cfd60fe322756b2e7772cc6c3b4841a81\": container with ID starting with d17a6464f2bdb469ae68d3613c098a3cfd60fe322756b2e7772cc6c3b4841a81 not found: ID does not exist"
Oct 31 04:00:48.023982 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:48.023944    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"]
Oct 31 04:00:48.075499 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:48.075466    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-b4pnm"]
Oct 31 04:00:49.539408 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:00:49.539372    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="136c00fa-e898-4bfd-9b48-9241fba6740e" path="/var/lib/kubelet/pods/136c00fa-e898-4bfd-9b48-9241fba6740e/volumes"
Oct 31 04:01:05.314543 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:01:05.314507    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:01:35.381739 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:01:35.381694    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:02:05.313347 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:02:05.313229    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:02:35.313318 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:02:35.313251    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:02:57.480159 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:02:57.480124    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"]
Oct 31 04:02:57.480624 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:02:57.480357    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp" podUID="6846468a-e677-43db-82e4-24f28fd7a0f6" containerName="istio-proxy" containerID="cri-o://6c1759996cbe947bb55870f4252a70a1f6bf8e4be459c4872d39ab8f3c0b0478" gracePeriod=30
Oct 31 04:02:58.556881 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:02:58.556849    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"]
Oct 31 04:02:58.557375 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:02:58.557056    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp" podUID="37196bbd-c341-4697-89c0-04ccffdfec09" containerName="istio-proxy" containerID="cri-o://8199d38beedaac2403f85bd8805366b90f91e0c8bd8b4f9c8e249f1dc7f4f607" gracePeriod=30
Oct 31 04:02:58.723567 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:02:58.723532    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"]
Oct 31 04:02:58.723950 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:02:58.723906    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84" podUID="8215d8db-0143-4917-97a5-091ba6bd2f6c" containerName="istio-proxy" containerID="cri-o://c8b8e1b5bdd4f00bc32f20408fa2755a3ed00383a9693952759e4a98fdba08e2" gracePeriod=30
Oct 31 04:02:58.775232 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:02:58.775185    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:02:58.775525 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:02:58.775486    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp" podUID="6846468a-e677-43db-82e4-24f28fd7a0f6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:03:02.809787 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.809756    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 04:03:02.902632 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.902593    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-workload-certs\") pod \"6846468a-e677-43db-82e4-24f28fd7a0f6\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") "
Oct 31 04:03:02.902830 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.902690    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-workload-socket\") pod \"6846468a-e677-43db-82e4-24f28fd7a0f6\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") "
Oct 31 04:03:02.902830 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.902722    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-credential-socket\") pod \"6846468a-e677-43db-82e4-24f28fd7a0f6\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") "
Oct 31 04:03:02.902830 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.902779    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6846468a-e677-43db-82e4-24f28fd7a0f6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6846468a-e677-43db-82e4-24f28fd7a0f6-istiod-ca-cert\") pod \"6846468a-e677-43db-82e4-24f28fd7a0f6\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") "
Oct 31 04:03:02.902830 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.902812    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-data\") pod \"6846468a-e677-43db-82e4-24f28fd7a0f6\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") "
Oct 31 04:03:02.903043 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.902878    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-workload-certs" pod "6846468a-e677-43db-82e4-24f28fd7a0f6" (UID: "6846468a-e677-43db-82e4-24f28fd7a0f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:02.903043 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.902898    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-token\" (UniqueName: \"kubernetes.io/projected/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-token\") pod \"6846468a-e677-43db-82e4-24f28fd7a0f6\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") "
Oct 31 04:03:02.903043 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.902928    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6846468a-e677-43db-82e4-24f28fd7a0f6-kube-api-access-pbtt7\" (UniqueName: \"kubernetes.io/projected/6846468a-e677-43db-82e4-24f28fd7a0f6-kube-api-access-pbtt7\") pod \"6846468a-e677-43db-82e4-24f28fd7a0f6\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") "
Oct 31 04:03:02.903043 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.903009    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-envoy\") pod \"6846468a-e677-43db-82e4-24f28fd7a0f6\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") "
Oct 31 04:03:02.903043 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.902993    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-workload-socket" pod "6846468a-e677-43db-82e4-24f28fd7a0f6" (UID: "6846468a-e677-43db-82e4-24f28fd7a0f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:02.903313 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.903054    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-data" pod "6846468a-e677-43db-82e4-24f28fd7a0f6" (UID: "6846468a-e677-43db-82e4-24f28fd7a0f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:02.903313 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.903083    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-podinfo\") pod \"6846468a-e677-43db-82e4-24f28fd7a0f6\" (UID: \"6846468a-e677-43db-82e4-24f28fd7a0f6\") "
Oct 31 04:03:02.903313 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.903080    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-credential-socket" pod "6846468a-e677-43db-82e4-24f28fd7a0f6" (UID: "6846468a-e677-43db-82e4-24f28fd7a0f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:02.903313 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.903158    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6846468a-e677-43db-82e4-24f28fd7a0f6-istiod-ca-cert" pod "6846468a-e677-43db-82e4-24f28fd7a0f6" (UID: "6846468a-e677-43db-82e4-24f28fd7a0f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:03:02.903708 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.903632    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:02.903708 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.903660    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:02.903708 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.903669    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:02.903708 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.903677    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6846468a-e677-43db-82e4-24f28fd7a0f6-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:02.903708 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.903689    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:02.905384 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.905353    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-podinfo" pod "6846468a-e677-43db-82e4-24f28fd7a0f6" (UID: "6846468a-e677-43db-82e4-24f28fd7a0f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:03:02.905487 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.905408    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-token" pod "6846468a-e677-43db-82e4-24f28fd7a0f6" (UID: "6846468a-e677-43db-82e4-24f28fd7a0f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:03:02.905798 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.905781    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-envoy" pod "6846468a-e677-43db-82e4-24f28fd7a0f6" (UID: "6846468a-e677-43db-82e4-24f28fd7a0f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:02.905798 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.905787    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6846468a-e677-43db-82e4-24f28fd7a0f6-kube-api-access-pbtt7" pod "6846468a-e677-43db-82e4-24f28fd7a0f6" (UID: "6846468a-e677-43db-82e4-24f28fd7a0f6"). InnerVolumeSpecName "kube-api-access-pbtt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:03:02.948865 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.948833    4926 generic.go:356] "Generic (PLEG): container finished" podID="6846468a-e677-43db-82e4-24f28fd7a0f6" containerID="6c1759996cbe947bb55870f4252a70a1f6bf8e4be459c4872d39ab8f3c0b0478" exitCode=0
Oct 31 04:03:02.949035 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.948887    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"
Oct 31 04:03:02.949035 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.948916    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp" event={"ID":"6846468a-e677-43db-82e4-24f28fd7a0f6","Type":"ContainerDied","Data":"6c1759996cbe947bb55870f4252a70a1f6bf8e4be459c4872d39ab8f3c0b0478"}
Oct 31 04:03:02.949035 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.948981    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp" event={"ID":"6846468a-e677-43db-82e4-24f28fd7a0f6","Type":"ContainerDied","Data":"dc23eae2528569419ea382de9754a9e4cb6df79e32ed9be3dd593ab546c1b92a"}
Oct 31 04:03:02.949035 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.949009    4926 scope.go:122] "RemoveContainer" containerID="6c1759996cbe947bb55870f4252a70a1f6bf8e4be459c4872d39ab8f3c0b0478"
Oct 31 04:03:02.973158 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.973133    4926 scope.go:122] "RemoveContainer" containerID="6c1759996cbe947bb55870f4252a70a1f6bf8e4be459c4872d39ab8f3c0b0478"
Oct 31 04:03:02.973509 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:03:02.973483    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c1759996cbe947bb55870f4252a70a1f6bf8e4be459c4872d39ab8f3c0b0478\": container with ID starting with 6c1759996cbe947bb55870f4252a70a1f6bf8e4be459c4872d39ab8f3c0b0478 not found: ID does not exist" containerID="6c1759996cbe947bb55870f4252a70a1f6bf8e4be459c4872d39ab8f3c0b0478"
Oct 31 04:03:02.973603 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:02.973521    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c1759996cbe947bb55870f4252a70a1f6bf8e4be459c4872d39ab8f3c0b0478"} err="failed to get container status \"6c1759996cbe947bb55870f4252a70a1f6bf8e4be459c4872d39ab8f3c0b0478\": rpc error: code = NotFound desc = could not find container \"6c1759996cbe947bb55870f4252a70a1f6bf8e4be459c4872d39ab8f3c0b0478\": container with ID starting with 6c1759996cbe947bb55870f4252a70a1f6bf8e4be459c4872d39ab8f3c0b0478 not found: ID does not exist"
Oct 31 04:03:03.004922 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.004882    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:03.004922 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.004919    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbtt7\" (UniqueName: \"kubernetes.io/projected/6846468a-e677-43db-82e4-24f28fd7a0f6-kube-api-access-pbtt7\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:03.005130 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.004936    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:03.005130 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.004950    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6846468a-e677-43db-82e4-24f28fd7a0f6-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:03.118553 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.118478    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"]
Oct 31 04:03:03.170844 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.170812    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-q7fvp"]
Oct 31 04:03:03.547908 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.547871    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6846468a-e677-43db-82e4-24f28fd7a0f6" path="/var/lib/kubelet/pods/6846468a-e677-43db-82e4-24f28fd7a0f6/volumes"
Oct 31 04:03:03.861057 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.861023    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 04:03:03.917452 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.917419    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-credential-socket\") pod \"37196bbd-c341-4697-89c0-04ccffdfec09\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") "
Oct 31 04:03:03.917624 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.917485    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-workload-socket\") pod \"37196bbd-c341-4697-89c0-04ccffdfec09\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") "
Oct 31 04:03:03.917624 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.917568    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37196bbd-c341-4697-89c0-04ccffdfec09-istio-token\" (UniqueName: \"kubernetes.io/projected/37196bbd-c341-4697-89c0-04ccffdfec09-istio-token\") pod \"37196bbd-c341-4697-89c0-04ccffdfec09\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") "
Oct 31 04:03:03.917753 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.917665    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-workload-certs\") pod \"37196bbd-c341-4697-89c0-04ccffdfec09\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") "
Oct 31 04:03:03.917753 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.917717    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-istio-envoy\") pod \"37196bbd-c341-4697-89c0-04ccffdfec09\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") "
Oct 31 04:03:03.917753 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.917731    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-credential-socket" pod "37196bbd-c341-4697-89c0-04ccffdfec09" (UID: "37196bbd-c341-4697-89c0-04ccffdfec09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:03.917918 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.917768    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/37196bbd-c341-4697-89c0-04ccffdfec09-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37196bbd-c341-4697-89c0-04ccffdfec09-istio-podinfo\") pod \"37196bbd-c341-4697-89c0-04ccffdfec09\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") "
Oct 31 04:03:03.917918 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.917802    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37196bbd-c341-4697-89c0-04ccffdfec09-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37196bbd-c341-4697-89c0-04ccffdfec09-istiod-ca-cert\") pod \"37196bbd-c341-4697-89c0-04ccffdfec09\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") "
Oct 31 04:03:03.917918 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.917835    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-istio-data\") pod \"37196bbd-c341-4697-89c0-04ccffdfec09\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") "
Oct 31 04:03:03.917918 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.917861    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-workload-socket" pod "37196bbd-c341-4697-89c0-04ccffdfec09" (UID: "37196bbd-c341-4697-89c0-04ccffdfec09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:03.918128 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.917947    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37196bbd-c341-4697-89c0-04ccffdfec09-kube-api-access-ft25p\" (UniqueName: \"kubernetes.io/projected/37196bbd-c341-4697-89c0-04ccffdfec09-kube-api-access-ft25p\") pod \"37196bbd-c341-4697-89c0-04ccffdfec09\" (UID: \"37196bbd-c341-4697-89c0-04ccffdfec09\") "
Oct 31 04:03:03.918318 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.918242    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-workload-certs" pod "37196bbd-c341-4697-89c0-04ccffdfec09" (UID: "37196bbd-c341-4697-89c0-04ccffdfec09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:03.919497 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.918712    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:03.919497 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.918742    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:03.919497 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.918773    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:03.919497 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.919017    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37196bbd-c341-4697-89c0-04ccffdfec09-istiod-ca-cert" pod "37196bbd-c341-4697-89c0-04ccffdfec09" (UID: "37196bbd-c341-4697-89c0-04ccffdfec09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:03:03.919497 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.919165    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-istio-data" pod "37196bbd-c341-4697-89c0-04ccffdfec09" (UID: "37196bbd-c341-4697-89c0-04ccffdfec09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:03.922455 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.922317    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-istio-envoy" pod "37196bbd-c341-4697-89c0-04ccffdfec09" (UID: "37196bbd-c341-4697-89c0-04ccffdfec09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:03.923887 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.923848    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37196bbd-c341-4697-89c0-04ccffdfec09-kube-api-access-ft25p" pod "37196bbd-c341-4697-89c0-04ccffdfec09" (UID: "37196bbd-c341-4697-89c0-04ccffdfec09"). InnerVolumeSpecName "kube-api-access-ft25p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:03:03.924023 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.924002    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37196bbd-c341-4697-89c0-04ccffdfec09-istio-podinfo" pod "37196bbd-c341-4697-89c0-04ccffdfec09" (UID: "37196bbd-c341-4697-89c0-04ccffdfec09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:03:03.924956 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.924934    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37196bbd-c341-4697-89c0-04ccffdfec09-istio-token" pod "37196bbd-c341-4697-89c0-04ccffdfec09" (UID: "37196bbd-c341-4697-89c0-04ccffdfec09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:03:03.960984 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.960946    4926 generic.go:356] "Generic (PLEG): container finished" podID="37196bbd-c341-4697-89c0-04ccffdfec09" containerID="8199d38beedaac2403f85bd8805366b90f91e0c8bd8b4f9c8e249f1dc7f4f607" exitCode=0
Oct 31 04:03:03.961163 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.961037    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp" event={"ID":"37196bbd-c341-4697-89c0-04ccffdfec09","Type":"ContainerDied","Data":"8199d38beedaac2403f85bd8805366b90f91e0c8bd8b4f9c8e249f1dc7f4f607"}
Oct 31 04:03:03.961163 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.961081    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp" event={"ID":"37196bbd-c341-4697-89c0-04ccffdfec09","Type":"ContainerDied","Data":"7667813a4ce3cb3aba10d83e2e41dd13db7ac3a42b31ded36c640f6d0fdfef9f"}
Oct 31 04:03:03.961163 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.961101    4926 scope.go:122] "RemoveContainer" containerID="8199d38beedaac2403f85bd8805366b90f91e0c8bd8b4f9c8e249f1dc7f4f607"
Oct 31 04:03:03.961372 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.961215    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"
Oct 31 04:03:03.964899 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.964883    4926 generic.go:356] "Generic (PLEG): container finished" podID="8215d8db-0143-4917-97a5-091ba6bd2f6c" containerID="c8b8e1b5bdd4f00bc32f20408fa2755a3ed00383a9693952759e4a98fdba08e2" exitCode=0
Oct 31 04:03:03.965037 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.964985    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84" event={"ID":"8215d8db-0143-4917-97a5-091ba6bd2f6c","Type":"ContainerDied","Data":"c8b8e1b5bdd4f00bc32f20408fa2755a3ed00383a9693952759e4a98fdba08e2"}
Oct 31 04:03:03.976125 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.976102    4926 scope.go:122] "RemoveContainer" containerID="8199d38beedaac2403f85bd8805366b90f91e0c8bd8b4f9c8e249f1dc7f4f607"
Oct 31 04:03:03.976412 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:03:03.976392    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8199d38beedaac2403f85bd8805366b90f91e0c8bd8b4f9c8e249f1dc7f4f607\": container with ID starting with 8199d38beedaac2403f85bd8805366b90f91e0c8bd8b4f9c8e249f1dc7f4f607 not found: ID does not exist" containerID="8199d38beedaac2403f85bd8805366b90f91e0c8bd8b4f9c8e249f1dc7f4f607"
Oct 31 04:03:03.976485 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.976420    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8199d38beedaac2403f85bd8805366b90f91e0c8bd8b4f9c8e249f1dc7f4f607"} err="failed to get container status \"8199d38beedaac2403f85bd8805366b90f91e0c8bd8b4f9c8e249f1dc7f4f607\": rpc error: code = NotFound desc = could not find container \"8199d38beedaac2403f85bd8805366b90f91e0c8bd8b4f9c8e249f1dc7f4f607\": container with ID starting with 8199d38beedaac2403f85bd8805366b90f91e0c8bd8b4f9c8e249f1dc7f4f607 not found: ID does not exist"
Oct 31 04:03:03.997995 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:03.994479    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 04:03:04.020592 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.020559    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:04.020592 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.020593    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37196bbd-c341-4697-89c0-04ccffdfec09-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:04.020783 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.020609    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37196bbd-c341-4697-89c0-04ccffdfec09-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:04.020783 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.020623    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37196bbd-c341-4697-89c0-04ccffdfec09-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:04.020783 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.020641    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft25p\" (UniqueName: \"kubernetes.io/projected/37196bbd-c341-4697-89c0-04ccffdfec09-kube-api-access-ft25p\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:04.020783 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.020656    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37196bbd-c341-4697-89c0-04ccffdfec09-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:04.106072 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.105985    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"]
Oct 31 04:03:04.108958 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.108932    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-bblrp"]
Oct 31 04:03:04.121905 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.121875    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-token\" (UniqueName: \"kubernetes.io/projected/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-token\") pod \"8215d8db-0143-4917-97a5-091ba6bd2f6c\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") "
Oct 31 04:03:04.122026 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.121982    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-data\") pod \"8215d8db-0143-4917-97a5-091ba6bd2f6c\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") "
Oct 31 04:03:04.122081 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.122038    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8215d8db-0143-4917-97a5-091ba6bd2f6c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8215d8db-0143-4917-97a5-091ba6bd2f6c-istiod-ca-cert\") pod \"8215d8db-0143-4917-97a5-091ba6bd2f6c\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") "
Oct 31 04:03:04.122141 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.122111    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-podinfo\") pod \"8215d8db-0143-4917-97a5-091ba6bd2f6c\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") "
Oct 31 04:03:04.122238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.122222    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-workload-certs\") pod \"8215d8db-0143-4917-97a5-091ba6bd2f6c\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") "
Oct 31 04:03:04.122362 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.122344    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8215d8db-0143-4917-97a5-091ba6bd2f6c-kube-api-access-pbwwk\" (UniqueName: \"kubernetes.io/projected/8215d8db-0143-4917-97a5-091ba6bd2f6c-kube-api-access-pbwwk\") pod \"8215d8db-0143-4917-97a5-091ba6bd2f6c\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") "
Oct 31 04:03:04.122425 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.122358    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-data" pod "8215d8db-0143-4917-97a5-091ba6bd2f6c" (UID: "8215d8db-0143-4917-97a5-091ba6bd2f6c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:04.122425 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.122387    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-credential-socket\") pod \"8215d8db-0143-4917-97a5-091ba6bd2f6c\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") "
Oct 31 04:03:04.122538 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.122482    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-envoy\") pod \"8215d8db-0143-4917-97a5-091ba6bd2f6c\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") "
Oct 31 04:03:04.122538 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.122504    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-workload-certs" pod "8215d8db-0143-4917-97a5-091ba6bd2f6c" (UID: "8215d8db-0143-4917-97a5-091ba6bd2f6c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:04.122647 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.122549    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-workload-socket\") pod \"8215d8db-0143-4917-97a5-091ba6bd2f6c\" (UID: \"8215d8db-0143-4917-97a5-091ba6bd2f6c\") "
Oct 31 04:03:04.122647 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.122590    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-credential-socket" pod "8215d8db-0143-4917-97a5-091ba6bd2f6c" (UID: "8215d8db-0143-4917-97a5-091ba6bd2f6c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:04.122749 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.122647    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8215d8db-0143-4917-97a5-091ba6bd2f6c-istiod-ca-cert" pod "8215d8db-0143-4917-97a5-091ba6bd2f6c" (UID: "8215d8db-0143-4917-97a5-091ba6bd2f6c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:03:04.122793 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.122756    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-workload-socket" pod "8215d8db-0143-4917-97a5-091ba6bd2f6c" (UID: "8215d8db-0143-4917-97a5-091ba6bd2f6c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:04.123497 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.123470    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:04.123613 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.123500    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:04.123613 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.123517    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:04.123613 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.123532    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8215d8db-0143-4917-97a5-091ba6bd2f6c-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:04.123613 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.123545    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:04.124678 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.124649    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-podinfo" pod "8215d8db-0143-4917-97a5-091ba6bd2f6c" (UID: "8215d8db-0143-4917-97a5-091ba6bd2f6c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:03:04.124756 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.124691    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-envoy" pod "8215d8db-0143-4917-97a5-091ba6bd2f6c" (UID: "8215d8db-0143-4917-97a5-091ba6bd2f6c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:04.124756 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.124716    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8215d8db-0143-4917-97a5-091ba6bd2f6c-kube-api-access-pbwwk" pod "8215d8db-0143-4917-97a5-091ba6bd2f6c" (UID: "8215d8db-0143-4917-97a5-091ba6bd2f6c"). InnerVolumeSpecName "kube-api-access-pbwwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:03:04.124843 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.124757    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-token" pod "8215d8db-0143-4917-97a5-091ba6bd2f6c" (UID: "8215d8db-0143-4917-97a5-091ba6bd2f6c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:03:04.225387 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.225349    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:04.225387 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.225382    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:04.225387 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.225392    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbwwk\" (UniqueName: \"kubernetes.io/projected/8215d8db-0143-4917-97a5-091ba6bd2f6c-kube-api-access-pbwwk\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:04.225638 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.225403    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8215d8db-0143-4917-97a5-091ba6bd2f6c-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:04.974311 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.974256    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"
Oct 31 04:03:04.974311 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.974299    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84" event={"ID":"8215d8db-0143-4917-97a5-091ba6bd2f6c","Type":"ContainerDied","Data":"152d668e67da1d958ae6314874d2446fe0752e623bccf69da33dec6540aec9e8"}
Oct 31 04:03:04.974825 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:04.974375    4926 scope.go:122] "RemoveContainer" containerID="c8b8e1b5bdd4f00bc32f20408fa2755a3ed00383a9693952759e4a98fdba08e2"
Oct 31 04:03:05.147184 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:05.147154    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"]
Oct 31 04:03:05.198125 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:05.198089    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-qql84"]
Oct 31 04:03:05.314205 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:03:05.314123    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:03:05.539456 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:05.539423    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37196bbd-c341-4697-89c0-04ccffdfec09" path="/var/lib/kubelet/pods/37196bbd-c341-4697-89c0-04ccffdfec09/volumes"
Oct 31 04:03:05.539941 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:05.539927    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8215d8db-0143-4917-97a5-091ba6bd2f6c" path="/var/lib/kubelet/pods/8215d8db-0143-4917-97a5-091ba6bd2f6c/volumes"
Oct 31 04:03:23.769226 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:23.769199    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:03:23.773113 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:23.773089    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:03:35.313680 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:03:35.313640    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:03:40.849894 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:40.849858    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"]
Oct 31 04:03:40.850417 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:40.850124    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq" podUID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" containerName="istio-proxy" containerID="cri-o://b54c8c42696db542485cff9251d4148060b18c0a541a2e8be8d08d1e477a1184" gracePeriod=30
Oct 31 04:03:46.144862 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.144836    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 04:03:46.238876 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.238844    4926 generic.go:356] "Generic (PLEG): container finished" podID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" containerID="b54c8c42696db542485cff9251d4148060b18c0a541a2e8be8d08d1e477a1184" exitCode=0
Oct 31 04:03:46.239040 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.238895    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"
Oct 31 04:03:46.239040 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.238925    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq" event={"ID":"b8e4ac44-94a2-480e-a2d9-9d41023b06ea","Type":"ContainerDied","Data":"b54c8c42696db542485cff9251d4148060b18c0a541a2e8be8d08d1e477a1184"}
Oct 31 04:03:46.239040 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.239015    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq" event={"ID":"b8e4ac44-94a2-480e-a2d9-9d41023b06ea","Type":"ContainerDied","Data":"e2e33a795f6cee259b996137f8ee0563af33718c0201b5755e8cc854af72659b"}
Oct 31 04:03:46.239140 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.239042    4926 scope.go:122] "RemoveContainer" containerID="b54c8c42696db542485cff9251d4148060b18c0a541a2e8be8d08d1e477a1184"
Oct 31 04:03:46.248843 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.248826    4926 scope.go:122] "RemoveContainer" containerID="15814adf9c0abaad7eb49d179b1675d5a16b95b0eead5a7a5d4217a87ade627c"
Oct 31 04:03:46.254545 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.254519    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-credential-socket\") pod \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") "
Oct 31 04:03:46.254697 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.254658    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-token\" (UniqueName: \"kubernetes.io/projected/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-token\") pod \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") "
Oct 31 04:03:46.254812 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.254786    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-workload-socket\") pod \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") "
Oct 31 04:03:46.254880 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.254821    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-envoy\") pod \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") "
Oct 31 04:03:46.254880 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.254843    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-credential-socket" pod "b8e4ac44-94a2-480e-a2d9-9d41023b06ea" (UID: "b8e4ac44-94a2-480e-a2d9-9d41023b06ea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:46.254880 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.254867    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-kube-api-access-xcdqx\" (UniqueName: \"kubernetes.io/projected/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-kube-api-access-xcdqx\") pod \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") "
Oct 31 04:03:46.255047 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.254916    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istiod-ca-cert\") pod \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") "
Oct 31 04:03:46.255047 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.254944    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-data\") pod \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") "
Oct 31 04:03:46.255047 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.255014    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-podinfo\") pod \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") "
Oct 31 04:03:46.255214 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.255056    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-workload-certs\") pod \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\" (UID: \"b8e4ac44-94a2-480e-a2d9-9d41023b06ea\") "
Oct 31 04:03:46.255478 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.255453    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-workload-socket" pod "b8e4ac44-94a2-480e-a2d9-9d41023b06ea" (UID: "b8e4ac44-94a2-480e-a2d9-9d41023b06ea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:46.255777 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.255717    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:46.255777 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.255747    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:46.255777 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.255743    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istiod-ca-cert" pod "b8e4ac44-94a2-480e-a2d9-9d41023b06ea" (UID: "b8e4ac44-94a2-480e-a2d9-9d41023b06ea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:03:46.258196 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.256234    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-data" pod "b8e4ac44-94a2-480e-a2d9-9d41023b06ea" (UID: "b8e4ac44-94a2-480e-a2d9-9d41023b06ea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:46.258196 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.257437    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-envoy" pod "b8e4ac44-94a2-480e-a2d9-9d41023b06ea" (UID: "b8e4ac44-94a2-480e-a2d9-9d41023b06ea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:46.258196 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.257618    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-token" pod "b8e4ac44-94a2-480e-a2d9-9d41023b06ea" (UID: "b8e4ac44-94a2-480e-a2d9-9d41023b06ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:03:46.258196 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.258024    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-workload-certs" pod "b8e4ac44-94a2-480e-a2d9-9d41023b06ea" (UID: "b8e4ac44-94a2-480e-a2d9-9d41023b06ea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:03:46.259683 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.259664    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-podinfo" pod "b8e4ac44-94a2-480e-a2d9-9d41023b06ea" (UID: "b8e4ac44-94a2-480e-a2d9-9d41023b06ea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:03:46.259790 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.259692    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-kube-api-access-xcdqx" pod "b8e4ac44-94a2-480e-a2d9-9d41023b06ea" (UID: "b8e4ac44-94a2-480e-a2d9-9d41023b06ea"). InnerVolumeSpecName "kube-api-access-xcdqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:03:46.260444 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.260424    4926 scope.go:122] "RemoveContainer" containerID="b54c8c42696db542485cff9251d4148060b18c0a541a2e8be8d08d1e477a1184"
Oct 31 04:03:46.260750 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:03:46.260725    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b54c8c42696db542485cff9251d4148060b18c0a541a2e8be8d08d1e477a1184\": container with ID starting with b54c8c42696db542485cff9251d4148060b18c0a541a2e8be8d08d1e477a1184 not found: ID does not exist" containerID="b54c8c42696db542485cff9251d4148060b18c0a541a2e8be8d08d1e477a1184"
Oct 31 04:03:46.260832 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.260763    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b54c8c42696db542485cff9251d4148060b18c0a541a2e8be8d08d1e477a1184"} err="failed to get container status \"b54c8c42696db542485cff9251d4148060b18c0a541a2e8be8d08d1e477a1184\": rpc error: code = NotFound desc = could not find container \"b54c8c42696db542485cff9251d4148060b18c0a541a2e8be8d08d1e477a1184\": container with ID starting with b54c8c42696db542485cff9251d4148060b18c0a541a2e8be8d08d1e477a1184 not found: ID does not exist"
Oct 31 04:03:46.260832 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.260788    4926 scope.go:122] "RemoveContainer" containerID="15814adf9c0abaad7eb49d179b1675d5a16b95b0eead5a7a5d4217a87ade627c"
Oct 31 04:03:46.261059 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:03:46.261041    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15814adf9c0abaad7eb49d179b1675d5a16b95b0eead5a7a5d4217a87ade627c\": container with ID starting with 15814adf9c0abaad7eb49d179b1675d5a16b95b0eead5a7a5d4217a87ade627c not found: ID does not exist" containerID="15814adf9c0abaad7eb49d179b1675d5a16b95b0eead5a7a5d4217a87ade627c"
Oct 31 04:03:46.261102 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.261063    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15814adf9c0abaad7eb49d179b1675d5a16b95b0eead5a7a5d4217a87ade627c"} err="failed to get container status \"15814adf9c0abaad7eb49d179b1675d5a16b95b0eead5a7a5d4217a87ade627c\": rpc error: code = NotFound desc = could not find container \"15814adf9c0abaad7eb49d179b1675d5a16b95b0eead5a7a5d4217a87ade627c\": container with ID starting with 15814adf9c0abaad7eb49d179b1675d5a16b95b0eead5a7a5d4217a87ade627c not found: ID does not exist"
Oct 31 04:03:46.357736 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.357705    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:46.357736 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.357736    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcdqx\" (UniqueName: \"kubernetes.io/projected/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-kube-api-access-xcdqx\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:46.357976 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.357751    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:46.357976 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.357764    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:46.357976 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.357778    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:46.357976 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.357790    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:46.357976 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.357803    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8e4ac44-94a2-480e-a2d9-9d41023b06ea-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:03:46.377605 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.377529    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"]
Oct 31 04:03:46.725943 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.725914    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"]
Oct 31 04:03:46.729953 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:46.729927    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-22e20532-openshift-default-6b75468d65-flzlq"]
Oct 31 04:03:47.538808 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:47.538776    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" path="/var/lib/kubelet/pods/b8e4ac44-94a2-480e-a2d9-9d41023b06ea/volumes"
Oct 31 04:03:50.241293 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:50.231393    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kbw7q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:03:50.241293 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:50.231470    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q" podUID="a9e112a1-fb10-4386-9700-939a8c0892da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:03:50.605453 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:50.605360    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-qk79t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:03:50.605453 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:03:50.605418    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t" podUID="170d646b-12d9-47eb-88db-232869e6854e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:04:01.398570 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.398532    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj" podUID="a5ad5303-15e7-422a-a144-5598d90f33db" containerName="main" containerID="cri-o://12a1fc54829057d527705be7f2e3da7400144278df9c1dbc09d456b68f989c35" gracePeriod=45
Oct 31 04:04:01.678886 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.678854    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 04:04:01.747480 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.747443    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-model-cache\") pod \"a5ad5303-15e7-422a-a144-5598d90f33db\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") "
Oct 31 04:04:01.747665 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.747503    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-tmp-dir\") pod \"a5ad5303-15e7-422a-a144-5598d90f33db\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") "
Oct 31 04:04:01.747665 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.747564    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-dshm\") pod \"a5ad5303-15e7-422a-a144-5598d90f33db\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") "
Oct 31 04:04:01.747757 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.747690    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-home\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-home\") pod \"a5ad5303-15e7-422a-a144-5598d90f33db\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") "
Oct 31 04:04:01.747757 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.747745    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a5ad5303-15e7-422a-a144-5598d90f33db-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ad5303-15e7-422a-a144-5598d90f33db-tls-certs\") pod \"a5ad5303-15e7-422a-a144-5598d90f33db\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") "
Oct 31 04:04:01.747848 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.747767    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-tmp-dir" pod "a5ad5303-15e7-422a-a144-5598d90f33db" (UID: "a5ad5303-15e7-422a-a144-5598d90f33db"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:01.747848 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.747776    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-model-cache" pod "a5ad5303-15e7-422a-a144-5598d90f33db" (UID: "a5ad5303-15e7-422a-a144-5598d90f33db"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:01.747955 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.747867    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5ad5303-15e7-422a-a144-5598d90f33db-kube-api-access-rp6lk\" (UniqueName: \"kubernetes.io/projected/a5ad5303-15e7-422a-a144-5598d90f33db-kube-api-access-rp6lk\") pod \"a5ad5303-15e7-422a-a144-5598d90f33db\" (UID: \"a5ad5303-15e7-422a-a144-5598d90f33db\") "
Oct 31 04:04:01.747955 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.747943    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-home" pod "a5ad5303-15e7-422a-a144-5598d90f33db" (UID: "a5ad5303-15e7-422a-a144-5598d90f33db"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:01.748344 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.748321    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:01.748344 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.748343    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:01.748529 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.748356    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:01.750132 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.750108    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ad5303-15e7-422a-a144-5598d90f33db-tls-certs" pod "a5ad5303-15e7-422a-a144-5598d90f33db" (UID: "a5ad5303-15e7-422a-a144-5598d90f33db"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:04:01.750222 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.750131    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-dshm" pod "a5ad5303-15e7-422a-a144-5598d90f33db" (UID: "a5ad5303-15e7-422a-a144-5598d90f33db"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:01.750222 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.750164    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ad5303-15e7-422a-a144-5598d90f33db-kube-api-access-rp6lk" pod "a5ad5303-15e7-422a-a144-5598d90f33db" (UID: "a5ad5303-15e7-422a-a144-5598d90f33db"). InnerVolumeSpecName "kube-api-access-rp6lk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:04:01.850407 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.850355    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5ad5303-15e7-422a-a144-5598d90f33db-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:01.850407 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.850394    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp6lk\" (UniqueName: \"kubernetes.io/projected/a5ad5303-15e7-422a-a144-5598d90f33db-kube-api-access-rp6lk\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:01.850407 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:01.850409    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5ad5303-15e7-422a-a144-5598d90f33db-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:02.363799 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:02.363749    4926 generic.go:356] "Generic (PLEG): container finished" podID="a5ad5303-15e7-422a-a144-5598d90f33db" containerID="12a1fc54829057d527705be7f2e3da7400144278df9c1dbc09d456b68f989c35" exitCode=0
Oct 31 04:04:02.363979 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:02.363813    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"
Oct 31 04:04:02.363979 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:02.363819    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj" event={"ID":"a5ad5303-15e7-422a-a144-5598d90f33db","Type":"ContainerDied","Data":"12a1fc54829057d527705be7f2e3da7400144278df9c1dbc09d456b68f989c35"}
Oct 31 04:04:02.363979 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:02.363876    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj" event={"ID":"a5ad5303-15e7-422a-a144-5598d90f33db","Type":"ContainerDied","Data":"3a6b706d2259aca05793e0694e58f4f275b24041e8f7a21b5315293a4b15318c"}
Oct 31 04:04:02.363979 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:02.363903    4926 scope.go:122] "RemoveContainer" containerID="12a1fc54829057d527705be7f2e3da7400144278df9c1dbc09d456b68f989c35"
Oct 31 04:04:02.374378 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:02.374354    4926 scope.go:122] "RemoveContainer" containerID="12a1fc54829057d527705be7f2e3da7400144278df9c1dbc09d456b68f989c35"
Oct 31 04:04:02.374676 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:04:02.374654    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12a1fc54829057d527705be7f2e3da7400144278df9c1dbc09d456b68f989c35\": container with ID starting with 12a1fc54829057d527705be7f2e3da7400144278df9c1dbc09d456b68f989c35 not found: ID does not exist" containerID="12a1fc54829057d527705be7f2e3da7400144278df9c1dbc09d456b68f989c35"
Oct 31 04:04:02.374749 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:02.374690    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12a1fc54829057d527705be7f2e3da7400144278df9c1dbc09d456b68f989c35"} err="failed to get container status \"12a1fc54829057d527705be7f2e3da7400144278df9c1dbc09d456b68f989c35\": rpc error: code = NotFound desc = could not find container \"12a1fc54829057d527705be7f2e3da7400144278df9c1dbc09d456b68f989c35\": container with ID starting with 12a1fc54829057d527705be7f2e3da7400144278df9c1dbc09d456b68f989c35 not found: ID does not exist"
Oct 31 04:04:02.534243 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:02.534213    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"]
Oct 31 04:04:02.538249 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:02.538219    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-68aa3e/simulator-w4-68aa3e-kserve-6fc565d487-cwzwj"]
Oct 31 04:04:03.540355 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:03.540315    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5ad5303-15e7-422a-a144-5598d90f33db" path="/var/lib/kubelet/pods/a5ad5303-15e7-422a-a144-5598d90f33db/volumes"
Oct 31 04:04:05.314744 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:04:05.314710    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:04:26.555923 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.555889    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"]
Oct 31 04:04:26.556422 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556409    4926 cpu_manager.go:414] "containerMap: removing container" podUID="766c0067-9aba-4388-906d-171ff0dd333c" containerName="cleanup"
Oct 31 04:04:26.556422 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556423    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="766c0067-9aba-4388-906d-171ff0dd333c" containerName="cleanup"
Oct 31 04:04:26.556505 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556433    4926 cpu_manager.go:414] "containerMap: removing container" podUID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf" containerName="cleanup"
Oct 31 04:04:26.556505 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556439    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf" containerName="cleanup"
Oct 31 04:04:26.556505 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556446    4926 cpu_manager.go:414] "containerMap: removing container" podUID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" containerName="istio-proxy"
Oct 31 04:04:26.556505 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556450    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" containerName="istio-proxy"
Oct 31 04:04:26.556505 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556459    4926 cpu_manager.go:414] "containerMap: removing container" podUID="766c0067-9aba-4388-906d-171ff0dd333c" containerName="cleanup"
Oct 31 04:04:26.556505 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556463    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="766c0067-9aba-4388-906d-171ff0dd333c" containerName="cleanup"
Oct 31 04:04:26.556505 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556471    4926 cpu_manager.go:414] "containerMap: removing container" podUID="9dec67dd-f744-4a84-8c94-10e91694a5d5" containerName="cleanup"
Oct 31 04:04:26.556505 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556476    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9dec67dd-f744-4a84-8c94-10e91694a5d5" containerName="cleanup"
Oct 31 04:04:26.556505 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556482    4926 cpu_manager.go:414] "containerMap: removing container" podUID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf" containerName="cleanup"
Oct 31 04:04:26.556505 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556486    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf" containerName="cleanup"
Oct 31 04:04:26.556505 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556494    4926 cpu_manager.go:414] "containerMap: removing container" podUID="9dec67dd-f744-4a84-8c94-10e91694a5d5" containerName="cleanup"
Oct 31 04:04:26.556505 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556499    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9dec67dd-f744-4a84-8c94-10e91694a5d5" containerName="cleanup"
Oct 31 04:04:26.556505 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556506    4926 cpu_manager.go:414] "containerMap: removing container" podUID="136c00fa-e898-4bfd-9b48-9241fba6740e" containerName="istio-proxy"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556511    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="136c00fa-e898-4bfd-9b48-9241fba6740e" containerName="istio-proxy"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556518    4926 cpu_manager.go:414] "containerMap: removing container" podUID="6846468a-e677-43db-82e4-24f28fd7a0f6" containerName="istio-proxy"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556522    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6846468a-e677-43db-82e4-24f28fd7a0f6" containerName="istio-proxy"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556535    4926 cpu_manager.go:414] "containerMap: removing container" podUID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" containerName="istio-proxy"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556539    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" containerName="istio-proxy"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556546    4926 cpu_manager.go:414] "containerMap: removing container" podUID="37196bbd-c341-4697-89c0-04ccffdfec09" containerName="istio-proxy"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556550    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37196bbd-c341-4697-89c0-04ccffdfec09" containerName="istio-proxy"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556564    4926 cpu_manager.go:414] "containerMap: removing container" podUID="a5ad5303-15e7-422a-a144-5598d90f33db" containerName="main"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556568    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5ad5303-15e7-422a-a144-5598d90f33db" containerName="main"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556577    4926 cpu_manager.go:414] "containerMap: removing container" podUID="8215d8db-0143-4917-97a5-091ba6bd2f6c" containerName="istio-proxy"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556581    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8215d8db-0143-4917-97a5-091ba6bd2f6c" containerName="istio-proxy"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556590    4926 cpu_manager.go:414] "containerMap: removing container" podUID="9dec67dd-f744-4a84-8c94-10e91694a5d5" containerName="cleanup"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556594    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9dec67dd-f744-4a84-8c94-10e91694a5d5" containerName="cleanup"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556676    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="766c0067-9aba-4388-906d-171ff0dd333c" containerName="cleanup"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556684    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5ad5303-15e7-422a-a144-5598d90f33db" containerName="main"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556691    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="8215d8db-0143-4917-97a5-091ba6bd2f6c" containerName="istio-proxy"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556698    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="9dec67dd-f744-4a84-8c94-10e91694a5d5" containerName="cleanup"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556706    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="766c0067-9aba-4388-906d-171ff0dd333c" containerName="cleanup"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556711    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf" containerName="cleanup"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556716    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" containerName="istio-proxy"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556721    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="766c0067-9aba-4388-906d-171ff0dd333c" containerName="cleanup"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556728    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf" containerName="cleanup"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556733    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="9dec67dd-f744-4a84-8c94-10e91694a5d5" containerName="cleanup"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556737    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="136c00fa-e898-4bfd-9b48-9241fba6740e" containerName="istio-proxy"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556742    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="6846468a-e677-43db-82e4-24f28fd7a0f6" containerName="istio-proxy"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556749    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8e4ac44-94a2-480e-a2d9-9d41023b06ea" containerName="istio-proxy"
Oct 31 04:04:26.556871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.556754    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="37196bbd-c341-4697-89c0-04ccffdfec09" containerName="istio-proxy"
Oct 31 04:04:26.561535 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.561517    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:26.617327 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.617288    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"]
Oct 31 04:04:26.663684 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.663650    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-288b63-serving-cert\""
Oct 31 04:04:26.718385 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.718346    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9f51242-834d-4824-b9c9-05315f436249-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:26.718763 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.718736    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9f51242-834d-4824-b9c9-05315f436249-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:26.718906 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.718823    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dpk7\" (UniqueName: \"kubernetes.io/projected/c9f51242-834d-4824-b9c9-05315f436249-kube-api-access-5dpk7\") pod \"maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:26.718971 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.718922    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9f51242-834d-4824-b9c9-05315f436249-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:26.888840 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.888720    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9f51242-834d-4824-b9c9-05315f436249-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:26.931365 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.931261    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9f51242-834d-4824-b9c9-05315f436249-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:26.932104 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.931718    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dpk7\" (UniqueName: \"kubernetes.io/projected/c9f51242-834d-4824-b9c9-05315f436249-kube-api-access-5dpk7\") pod \"maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:26.932428 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.932408    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9f51242-834d-4824-b9c9-05315f436249-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:26.933151 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.933130    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9f51242-834d-4824-b9c9-05315f436249-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:26.936822 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.936800    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9f51242-834d-4824-b9c9-05315f436249-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:26.937736 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.937714    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9f51242-834d-4824-b9c9-05315f436249-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:26.994756 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:26.994722    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dpk7\" (UniqueName: \"kubernetes.io/projected/c9f51242-834d-4824-b9c9-05315f436249-kube-api-access-5dpk7\") pod \"maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:27.174759 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:27.174633    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:27.799806 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:04:27.799772    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9f51242_834d_4824_b9c9_05315f436249.slice/crio-f45bdb61fb5409ffab5ac58b383494258bed537979f5ec8df8adcd88ed9f38be WatchSource:0}: Error finding container f45bdb61fb5409ffab5ac58b383494258bed537979f5ec8df8adcd88ed9f38be: Status 404 returned error can't find the container with id f45bdb61fb5409ffab5ac58b383494258bed537979f5ec8df8adcd88ed9f38be
Oct 31 04:04:27.803242 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:27.803220    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"]
Oct 31 04:04:28.531368 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:28.531332    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp" event={"ID":"c9f51242-834d-4824-b9c9-05315f436249","Type":"ContainerStarted","Data":"f45bdb61fb5409ffab5ac58b383494258bed537979f5ec8df8adcd88ed9f38be"}
Oct 31 04:04:29.539057 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:29.539025    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:29.539489 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:29.539079    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp" event={"ID":"c9f51242-834d-4824-b9c9-05315f436249","Type":"ContainerStarted","Data":"cb1352404c0ff772720c1f7bfcb732792da74a2532b456725dc2685dcb63b7c4"}
Oct 31 04:04:29.709312 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:29.708226    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp" podStartSLOduration=2.793911902 podStartE2EDuration="3.70820422s" totalImagesPullingTime="914.292318ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:04:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:04:29.704187181 +0000 UTC m=+2166.289727443" watchObservedRunningTime="2026-10-31 04:04:29.70820422 +0000 UTC m=+2166.293744487"
Oct 31 04:04:35.315063 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:04:35.315018    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:04:35.555623 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:35.555593    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:04:43.763211 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:43.763180    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"]
Oct 31 04:04:43.763625 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:43.763416    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj" podUID="2524a945-cbb0-42b3-b2be-8cd43a1f1df9" containerName="istio-proxy" containerID="cri-o://cb6965d4a09e2cb6ebb2ed4108846014f4a0ae2b94048100b9761b88bdf7939e" gracePeriod=30
Oct 31 04:04:45.213030 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:45.212993    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:04:45.213561 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:45.213064    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj" podUID="2524a945-cbb0-42b3-b2be-8cd43a1f1df9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:04:49.039738 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.039677    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 04:04:49.096783 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.096750    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-envoy\") pod \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") "
Oct 31 04:04:49.096963 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.096816    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-workload-certs\") pod \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") "
Oct 31 04:04:49.096963 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.096857    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istiod-ca-cert\") pod \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") "
Oct 31 04:04:49.096963 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.096934    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-credential-socket\") pod \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") "
Oct 31 04:04:49.097141 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.096980    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-data\") pod \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") "
Oct 31 04:04:49.097141 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.097036    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-podinfo\") pod \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") "
Oct 31 04:04:49.097141 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.097077    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-token\" (UniqueName: \"kubernetes.io/projected/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-token\") pod \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") "
Oct 31 04:04:49.097337 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.097131    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-workload-certs" pod "2524a945-cbb0-42b3-b2be-8cd43a1f1df9" (UID: "2524a945-cbb0-42b3-b2be-8cd43a1f1df9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:49.097337 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.097179    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-kube-api-access-cxvzq\" (UniqueName: \"kubernetes.io/projected/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-kube-api-access-cxvzq\") pod \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") "
Oct 31 04:04:49.097337 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.097302    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-workload-socket\") pod \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\" (UID: \"2524a945-cbb0-42b3-b2be-8cd43a1f1df9\") "
Oct 31 04:04:49.097491 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.097329    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-data" pod "2524a945-cbb0-42b3-b2be-8cd43a1f1df9" (UID: "2524a945-cbb0-42b3-b2be-8cd43a1f1df9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:49.097491 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.097380    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-credential-socket" pod "2524a945-cbb0-42b3-b2be-8cd43a1f1df9" (UID: "2524a945-cbb0-42b3-b2be-8cd43a1f1df9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:49.097491 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.097384    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istiod-ca-cert" pod "2524a945-cbb0-42b3-b2be-8cd43a1f1df9" (UID: "2524a945-cbb0-42b3-b2be-8cd43a1f1df9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:04:49.097807 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.097664    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-workload-socket" pod "2524a945-cbb0-42b3-b2be-8cd43a1f1df9" (UID: "2524a945-cbb0-42b3-b2be-8cd43a1f1df9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:49.098189 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.098163    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.098323 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.098195    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.098323 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.098210    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.098323 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.098224    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.098323 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.098238    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.099475 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.099444    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-envoy" pod "2524a945-cbb0-42b3-b2be-8cd43a1f1df9" (UID: "2524a945-cbb0-42b3-b2be-8cd43a1f1df9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:49.099594 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.099471    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-kube-api-access-cxvzq" pod "2524a945-cbb0-42b3-b2be-8cd43a1f1df9" (UID: "2524a945-cbb0-42b3-b2be-8cd43a1f1df9"). InnerVolumeSpecName "kube-api-access-cxvzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:04:49.099594 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.099579    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-podinfo" pod "2524a945-cbb0-42b3-b2be-8cd43a1f1df9" (UID: "2524a945-cbb0-42b3-b2be-8cd43a1f1df9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:04:49.099682 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.099608    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-token" pod "2524a945-cbb0-42b3-b2be-8cd43a1f1df9" (UID: "2524a945-cbb0-42b3-b2be-8cd43a1f1df9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:04:49.199684 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.199651    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.199684 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.199678    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.199684 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.199688    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.199914 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.199697    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxvzq\" (UniqueName: \"kubernetes.io/projected/2524a945-cbb0-42b3-b2be-8cd43a1f1df9-kube-api-access-cxvzq\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:49.717868 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.717827    4926 generic.go:356] "Generic (PLEG): container finished" podID="2524a945-cbb0-42b3-b2be-8cd43a1f1df9" containerID="cb6965d4a09e2cb6ebb2ed4108846014f4a0ae2b94048100b9761b88bdf7939e" exitCode=0
Oct 31 04:04:49.718043 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.717884    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"
Oct 31 04:04:49.718043 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.717918    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj" event={"ID":"2524a945-cbb0-42b3-b2be-8cd43a1f1df9","Type":"ContainerDied","Data":"cb6965d4a09e2cb6ebb2ed4108846014f4a0ae2b94048100b9761b88bdf7939e"}
Oct 31 04:04:49.718043 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.717987    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj" event={"ID":"2524a945-cbb0-42b3-b2be-8cd43a1f1df9","Type":"ContainerDied","Data":"c344932a25e3d2a40170cf6c20799365426ea7347158d1d42db9cc21ad2eaa13"}
Oct 31 04:04:49.718043 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.718015    4926 scope.go:122] "RemoveContainer" containerID="cb6965d4a09e2cb6ebb2ed4108846014f4a0ae2b94048100b9761b88bdf7939e"
Oct 31 04:04:49.727670 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.727649    4926 scope.go:122] "RemoveContainer" containerID="cb6965d4a09e2cb6ebb2ed4108846014f4a0ae2b94048100b9761b88bdf7939e"
Oct 31 04:04:49.727999 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:04:49.727975    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb6965d4a09e2cb6ebb2ed4108846014f4a0ae2b94048100b9761b88bdf7939e\": container with ID starting with cb6965d4a09e2cb6ebb2ed4108846014f4a0ae2b94048100b9761b88bdf7939e not found: ID does not exist" containerID="cb6965d4a09e2cb6ebb2ed4108846014f4a0ae2b94048100b9761b88bdf7939e"
Oct 31 04:04:49.728057 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.728008    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb6965d4a09e2cb6ebb2ed4108846014f4a0ae2b94048100b9761b88bdf7939e"} err="failed to get container status \"cb6965d4a09e2cb6ebb2ed4108846014f4a0ae2b94048100b9761b88bdf7939e\": rpc error: code = NotFound desc = could not find container \"cb6965d4a09e2cb6ebb2ed4108846014f4a0ae2b94048100b9761b88bdf7939e\": container with ID starting with cb6965d4a09e2cb6ebb2ed4108846014f4a0ae2b94048100b9761b88bdf7939e not found: ID does not exist"
Oct 31 04:04:49.941586 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.941508    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"]
Oct 31 04:04:49.992642 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:49.992613    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-khlqj"]
Oct 31 04:04:51.539656 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:51.539619    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2524a945-cbb0-42b3-b2be-8cd43a1f1df9" path="/var/lib/kubelet/pods/2524a945-cbb0-42b3-b2be-8cd43a1f1df9/volumes"
Oct 31 04:04:54.446922 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:54.446885    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"]
Oct 31 04:04:54.447536 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:54.447141    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2" podUID="08979c91-eaa0-4650-ace2-eda96f061f1d" containerName="istio-proxy" containerID="cri-o://d689544d444ee0b479f8ab9b1a885a4b7516094cdeaae9c1fc2a457fa3a04c88" gracePeriod=30
Oct 31 04:04:59.728495 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.728435    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 04:04:59.796827 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.796798    4926 generic.go:356] "Generic (PLEG): container finished" podID="08979c91-eaa0-4650-ace2-eda96f061f1d" containerID="d689544d444ee0b479f8ab9b1a885a4b7516094cdeaae9c1fc2a457fa3a04c88" exitCode=0
Oct 31 04:04:59.796969 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.796865    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"
Oct 31 04:04:59.796969 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.796876    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2" event={"ID":"08979c91-eaa0-4650-ace2-eda96f061f1d","Type":"ContainerDied","Data":"d689544d444ee0b479f8ab9b1a885a4b7516094cdeaae9c1fc2a457fa3a04c88"}
Oct 31 04:04:59.796969 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.796946    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2" event={"ID":"08979c91-eaa0-4650-ace2-eda96f061f1d","Type":"ContainerDied","Data":"d45fc344f2b75ad8ddb4c284b5cacf2da4d9e1fbc967e8dfae961b6cc2066841"}
Oct 31 04:04:59.797114 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.796978    4926 scope.go:122] "RemoveContainer" containerID="d689544d444ee0b479f8ab9b1a885a4b7516094cdeaae9c1fc2a457fa3a04c88"
Oct 31 04:04:59.807918 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.807899    4926 scope.go:122] "RemoveContainer" containerID="64a90b70b5f5a03383070b3d5d437c91fec4f2c452f7b39622fabd9dcebb2d87"
Oct 31 04:04:59.817087 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.817070    4926 scope.go:122] "RemoveContainer" containerID="d689544d444ee0b479f8ab9b1a885a4b7516094cdeaae9c1fc2a457fa3a04c88"
Oct 31 04:04:59.817382 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:04:59.817361    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d689544d444ee0b479f8ab9b1a885a4b7516094cdeaae9c1fc2a457fa3a04c88\": container with ID starting with d689544d444ee0b479f8ab9b1a885a4b7516094cdeaae9c1fc2a457fa3a04c88 not found: ID does not exist" containerID="d689544d444ee0b479f8ab9b1a885a4b7516094cdeaae9c1fc2a457fa3a04c88"
Oct 31 04:04:59.817443 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.817394    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d689544d444ee0b479f8ab9b1a885a4b7516094cdeaae9c1fc2a457fa3a04c88"} err="failed to get container status \"d689544d444ee0b479f8ab9b1a885a4b7516094cdeaae9c1fc2a457fa3a04c88\": rpc error: code = NotFound desc = could not find container \"d689544d444ee0b479f8ab9b1a885a4b7516094cdeaae9c1fc2a457fa3a04c88\": container with ID starting with d689544d444ee0b479f8ab9b1a885a4b7516094cdeaae9c1fc2a457fa3a04c88 not found: ID does not exist"
Oct 31 04:04:59.817443 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.817415    4926 scope.go:122] "RemoveContainer" containerID="64a90b70b5f5a03383070b3d5d437c91fec4f2c452f7b39622fabd9dcebb2d87"
Oct 31 04:04:59.817664 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:04:59.817647    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64a90b70b5f5a03383070b3d5d437c91fec4f2c452f7b39622fabd9dcebb2d87\": container with ID starting with 64a90b70b5f5a03383070b3d5d437c91fec4f2c452f7b39622fabd9dcebb2d87 not found: ID does not exist" containerID="64a90b70b5f5a03383070b3d5d437c91fec4f2c452f7b39622fabd9dcebb2d87"
Oct 31 04:04:59.817720 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.817676    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64a90b70b5f5a03383070b3d5d437c91fec4f2c452f7b39622fabd9dcebb2d87"} err="failed to get container status \"64a90b70b5f5a03383070b3d5d437c91fec4f2c452f7b39622fabd9dcebb2d87\": rpc error: code = NotFound desc = could not find container \"64a90b70b5f5a03383070b3d5d437c91fec4f2c452f7b39622fabd9dcebb2d87\": container with ID starting with 64a90b70b5f5a03383070b3d5d437c91fec4f2c452f7b39622fabd9dcebb2d87 not found: ID does not exist"
Oct 31 04:04:59.842424 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.842396    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08979c91-eaa0-4650-ace2-eda96f061f1d-kube-api-access-k9ppn\" (UniqueName: \"kubernetes.io/projected/08979c91-eaa0-4650-ace2-eda96f061f1d-kube-api-access-k9ppn\") pod \"08979c91-eaa0-4650-ace2-eda96f061f1d\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") "
Oct 31 04:04:59.842538 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.842499    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-workload-certs\") pod \"08979c91-eaa0-4650-ace2-eda96f061f1d\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") "
Oct 31 04:04:59.842538 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.842532    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-envoy\") pod \"08979c91-eaa0-4650-ace2-eda96f061f1d\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") "
Oct 31 04:04:59.842654 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.842557    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-credential-socket\") pod \"08979c91-eaa0-4650-ace2-eda96f061f1d\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") "
Oct 31 04:04:59.842654 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.842635    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-data\") pod \"08979c91-eaa0-4650-ace2-eda96f061f1d\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") "
Oct 31 04:04:59.842758 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.842690    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-workload-socket\") pod \"08979c91-eaa0-4650-ace2-eda96f061f1d\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") "
Oct 31 04:04:59.842809 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.842761    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-podinfo\") pod \"08979c91-eaa0-4650-ace2-eda96f061f1d\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") "
Oct 31 04:04:59.842865 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.842807    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-workload-certs" pod "08979c91-eaa0-4650-ace2-eda96f061f1d" (UID: "08979c91-eaa0-4650-ace2-eda96f061f1d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:59.842865 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.842832    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/08979c91-eaa0-4650-ace2-eda96f061f1d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08979c91-eaa0-4650-ace2-eda96f061f1d-istiod-ca-cert\") pod \"08979c91-eaa0-4650-ace2-eda96f061f1d\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") "
Oct 31 04:04:59.842967 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.842863    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-token\" (UniqueName: \"kubernetes.io/projected/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-token\") pod \"08979c91-eaa0-4650-ace2-eda96f061f1d\" (UID: \"08979c91-eaa0-4650-ace2-eda96f061f1d\") "
Oct 31 04:04:59.843045 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.843010    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-data" pod "08979c91-eaa0-4650-ace2-eda96f061f1d" (UID: "08979c91-eaa0-4650-ace2-eda96f061f1d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:59.843367 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.843238    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-workload-socket" pod "08979c91-eaa0-4650-ace2-eda96f061f1d" (UID: "08979c91-eaa0-4650-ace2-eda96f061f1d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:59.843367 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.843250    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-credential-socket" pod "08979c91-eaa0-4650-ace2-eda96f061f1d" (UID: "08979c91-eaa0-4650-ace2-eda96f061f1d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:59.843367 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.843330    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08979c91-eaa0-4650-ace2-eda96f061f1d-istiod-ca-cert" pod "08979c91-eaa0-4650-ace2-eda96f061f1d" (UID: "08979c91-eaa0-4650-ace2-eda96f061f1d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:04:59.843592 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.843464    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:59.843592 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.843479    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:59.843592 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.843487    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/08979c91-eaa0-4650-ace2-eda96f061f1d-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:59.843592 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.843496    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:59.843592 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.843505    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:59.845184 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.845165    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-envoy" pod "08979c91-eaa0-4650-ace2-eda96f061f1d" (UID: "08979c91-eaa0-4650-ace2-eda96f061f1d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:04:59.845403 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.845248    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08979c91-eaa0-4650-ace2-eda96f061f1d-kube-api-access-k9ppn" pod "08979c91-eaa0-4650-ace2-eda96f061f1d" (UID: "08979c91-eaa0-4650-ace2-eda96f061f1d"). InnerVolumeSpecName "kube-api-access-k9ppn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:04:59.845403 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.845345    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-podinfo" pod "08979c91-eaa0-4650-ace2-eda96f061f1d" (UID: "08979c91-eaa0-4650-ace2-eda96f061f1d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:04:59.845403 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.845376    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-token" pod "08979c91-eaa0-4650-ace2-eda96f061f1d" (UID: "08979c91-eaa0-4650-ace2-eda96f061f1d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:04:59.944421 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.944391    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:59.944421 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.944416    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:59.944617 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.944426    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/08979c91-eaa0-4650-ace2-eda96f061f1d-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:04:59.944617 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:04:59.944447    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9ppn\" (UniqueName: \"kubernetes.io/projected/08979c91-eaa0-4650-ace2-eda96f061f1d-kube-api-access-k9ppn\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:00.224516 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:00.224486    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"]
Oct 31 04:05:00.227997 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:00.227972    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-b4a3a5-openshift-default-756d9c6785-6p9l2"]
Oct 31 04:05:01.548087 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:01.548042    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="08979c91-eaa0-4650-ace2-eda96f061f1d" path="/var/lib/kubelet/pods/08979c91-eaa0-4650-ace2-eda96f061f1d/volumes"
Oct 31 04:05:05.314236 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:05:05.314147    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:05:35.315190 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:05:35.315154    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:05:43.719152 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:43.719113    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"]
Oct 31 04:05:43.719679 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:43.719332    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv" podUID="362761bb-e08f-4109-8989-3acebfc6b762" containerName="istio-proxy" containerID="cri-o://45124883a5690f73fe168243314dfaf40f58ed2e28129b43f0d165e7c8835870" gracePeriod=30
Oct 31 04:05:43.828034 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:43.828002    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"]
Oct 31 04:05:43.828332 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:43.828248    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq" podUID="7d0301a0-5d7b-4305-b4dd-24539a745220" containerName="discovery" containerID="cri-o://e0effab5bf975bc5912bd7155f1c8b4d735adbd51adf4b16712109aa995a4845" gracePeriod=30
Oct 31 04:05:43.832912 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:43.832882    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"]
Oct 31 04:05:43.833160 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:43.833111    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92" podUID="88086a57-3749-41e6-8d6b-0fe95866237e" containerName="istio-proxy" containerID="cri-o://76f8e9a679714ae42f643032203cf655542e664add8fb0b0bb434be47c9cc263" gracePeriod=30
Oct 31 04:05:44.071959 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.071922    4926 generic.go:356] "Generic (PLEG): container finished" podID="7d0301a0-5d7b-4305-b4dd-24539a745220" containerID="e0effab5bf975bc5912bd7155f1c8b4d735adbd51adf4b16712109aa995a4845" exitCode=0
Oct 31 04:05:44.072115 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.071990    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq" event={"ID":"7d0301a0-5d7b-4305-b4dd-24539a745220","Type":"ContainerDied","Data":"e0effab5bf975bc5912bd7155f1c8b4d735adbd51adf4b16712109aa995a4845"}
Oct 31 04:05:44.121702 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.121678    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 04:05:44.234540 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.234511    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7d0301a0-5d7b-4305-b4dd-24539a745220-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d0301a0-5d7b-4305-b4dd-24539a745220-local-certs\") pod \"7d0301a0-5d7b-4305-b4dd-24539a745220\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") "
Oct 31 04:05:44.234726 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.234619    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-kubeconfig\") pod \"7d0301a0-5d7b-4305-b4dd-24539a745220\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") "
Oct 31 04:05:44.234726 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.234646    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-cacerts\" (UniqueName: \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-cacerts\") pod \"7d0301a0-5d7b-4305-b4dd-24539a745220\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") "
Oct 31 04:05:44.234726 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.234668    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-csr-dns-cert\") pod \"7d0301a0-5d7b-4305-b4dd-24539a745220\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") "
Oct 31 04:05:44.234913 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.234754    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-token\" (UniqueName: \"kubernetes.io/projected/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-token\") pod \"7d0301a0-5d7b-4305-b4dd-24539a745220\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") "
Oct 31 04:05:44.234972 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.234918    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-csr-ca-configmap\") pod \"7d0301a0-5d7b-4305-b4dd-24539a745220\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") "
Oct 31 04:05:44.235083 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.235061    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d0301a0-5d7b-4305-b4dd-24539a745220-kube-api-access-gkpzh\" (UniqueName: \"kubernetes.io/projected/7d0301a0-5d7b-4305-b4dd-24539a745220-kube-api-access-gkpzh\") pod \"7d0301a0-5d7b-4305-b4dd-24539a745220\" (UID: \"7d0301a0-5d7b-4305-b4dd-24539a745220\") "
Oct 31 04:05:44.235525 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.235495    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-csr-ca-configmap" pod "7d0301a0-5d7b-4305-b4dd-24539a745220" (UID: "7d0301a0-5d7b-4305-b4dd-24539a745220"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:05:44.236167 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.236141    4926 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-csr-ca-configmap\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:44.237037 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.237006    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d0301a0-5d7b-4305-b4dd-24539a745220-local-certs" pod "7d0301a0-5d7b-4305-b4dd-24539a745220" (UID: "7d0301a0-5d7b-4305-b4dd-24539a745220"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:44.237629 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.237602    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-cacerts" pod "7d0301a0-5d7b-4305-b4dd-24539a745220" (UID: "7d0301a0-5d7b-4305-b4dd-24539a745220"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:05:44.237713 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.237686    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d0301a0-5d7b-4305-b4dd-24539a745220-kube-api-access-gkpzh" pod "7d0301a0-5d7b-4305-b4dd-24539a745220" (UID: "7d0301a0-5d7b-4305-b4dd-24539a745220"). InnerVolumeSpecName "kube-api-access-gkpzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:05:44.237759 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.237708    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-csr-dns-cert" pod "7d0301a0-5d7b-4305-b4dd-24539a745220" (UID: "7d0301a0-5d7b-4305-b4dd-24539a745220"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:05:44.237759 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.237725    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-token" pod "7d0301a0-5d7b-4305-b4dd-24539a745220" (UID: "7d0301a0-5d7b-4305-b4dd-24539a745220"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:05:44.237827 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.237775    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-kubeconfig" pod "7d0301a0-5d7b-4305-b4dd-24539a745220" (UID: "7d0301a0-5d7b-4305-b4dd-24539a745220"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:05:44.337448 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.337372    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkpzh\" (UniqueName: \"kubernetes.io/projected/7d0301a0-5d7b-4305-b4dd-24539a745220-kube-api-access-gkpzh\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:44.337448 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.337398    4926 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/7d0301a0-5d7b-4305-b4dd-24539a745220-local-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:44.337448 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.337408    4926 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-kubeconfig\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:44.337448 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.337418    4926 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-cacerts\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:44.337448 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.337426    4926 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-csr-dns-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:44.337448 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.337435    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7d0301a0-5d7b-4305-b4dd-24539a745220-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:44.937991 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.937957    4926 patch_prober.go:29] interesting pod/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:05:44.938382 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:44.938021    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92" podUID="88086a57-3749-41e6-8d6b-0fe95866237e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:05:45.078960 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:45.078937    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"
Oct 31 04:05:45.079119 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:45.078975    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq" event={"ID":"7d0301a0-5d7b-4305-b4dd-24539a745220","Type":"ContainerDied","Data":"f91573770c7842735df02f09c70c74d7c3484abdbd6b93b7a9e4c670a1c2c078"}
Oct 31 04:05:45.079119 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:45.079043    4926 scope.go:122] "RemoveContainer" containerID="e0effab5bf975bc5912bd7155f1c8b4d735adbd51adf4b16712109aa995a4845"
Oct 31 04:05:45.250166 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:45.250126    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"]
Oct 31 04:05:45.301708 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:45.301673    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6ndmq"]
Oct 31 04:05:45.539422 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:45.539349    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d0301a0-5d7b-4305-b4dd-24539a745220" path="/var/lib/kubelet/pods/7d0301a0-5d7b-4305-b4dd-24539a745220/volumes"
Oct 31 04:05:45.799403 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:45.799308    4926 patch_prober.go:29] interesting pod/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:05:45.799403 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:45.799373    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv" podUID="362761bb-e08f-4109-8989-3acebfc6b762" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:05:49.007648 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.007623    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 04:05:49.101941 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.101914    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/362761bb-e08f-4109-8989-3acebfc6b762-istio-token\" (UniqueName: \"kubernetes.io/projected/362761bb-e08f-4109-8989-3acebfc6b762-istio-token\") pod \"362761bb-e08f-4109-8989-3acebfc6b762\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") "
Oct 31 04:05:49.102059 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.101983    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/362761bb-e08f-4109-8989-3acebfc6b762-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/362761bb-e08f-4109-8989-3acebfc6b762-istiod-ca-cert\") pod \"362761bb-e08f-4109-8989-3acebfc6b762\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") "
Oct 31 04:05:49.102059 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.102028    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-workload-certs\") pod \"362761bb-e08f-4109-8989-3acebfc6b762\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") "
Oct 31 04:05:49.102290 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.102246    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-istio-data\") pod \"362761bb-e08f-4109-8989-3acebfc6b762\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") "
Oct 31 04:05:49.102353 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.102299    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/362761bb-e08f-4109-8989-3acebfc6b762-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/362761bb-e08f-4109-8989-3acebfc6b762-istio-podinfo\") pod \"362761bb-e08f-4109-8989-3acebfc6b762\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") "
Oct 31 04:05:49.102353 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.102337    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-workload-socket\") pod \"362761bb-e08f-4109-8989-3acebfc6b762\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") "
Oct 31 04:05:49.102467 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.102369    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-credential-socket\") pod \"362761bb-e08f-4109-8989-3acebfc6b762\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") "
Oct 31 04:05:49.102467 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.102400    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-workload-certs" pod "362761bb-e08f-4109-8989-3acebfc6b762" (UID: "362761bb-e08f-4109-8989-3acebfc6b762"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:49.102467 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.102426    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/362761bb-e08f-4109-8989-3acebfc6b762-kube-api-access-qt6sm\" (UniqueName: \"kubernetes.io/projected/362761bb-e08f-4109-8989-3acebfc6b762-kube-api-access-qt6sm\") pod \"362761bb-e08f-4109-8989-3acebfc6b762\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") "
Oct 31 04:05:49.102641 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.102472    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-istio-envoy\") pod \"362761bb-e08f-4109-8989-3acebfc6b762\" (UID: \"362761bb-e08f-4109-8989-3acebfc6b762\") "
Oct 31 04:05:49.102641 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.102518    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-istio-data" pod "362761bb-e08f-4109-8989-3acebfc6b762" (UID: "362761bb-e08f-4109-8989-3acebfc6b762"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:49.102641 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.102439    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/362761bb-e08f-4109-8989-3acebfc6b762-istiod-ca-cert" pod "362761bb-e08f-4109-8989-3acebfc6b762" (UID: "362761bb-e08f-4109-8989-3acebfc6b762"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:05:49.102641 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.102627    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-workload-socket" pod "362761bb-e08f-4109-8989-3acebfc6b762" (UID: "362761bb-e08f-4109-8989-3acebfc6b762"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:49.102848 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.102758    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-credential-socket" pod "362761bb-e08f-4109-8989-3acebfc6b762" (UID: "362761bb-e08f-4109-8989-3acebfc6b762"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:49.103323 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.103119    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/362761bb-e08f-4109-8989-3acebfc6b762-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.103323 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.103145    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.103323 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.103159    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.103323 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.103172    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.103323 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.103187    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.104786 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.104759    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/362761bb-e08f-4109-8989-3acebfc6b762-kube-api-access-qt6sm" pod "362761bb-e08f-4109-8989-3acebfc6b762" (UID: "362761bb-e08f-4109-8989-3acebfc6b762"). InnerVolumeSpecName "kube-api-access-qt6sm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:05:49.104902 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.104847    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-istio-envoy" pod "362761bb-e08f-4109-8989-3acebfc6b762" (UID: "362761bb-e08f-4109-8989-3acebfc6b762"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:49.104964 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.104932    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/362761bb-e08f-4109-8989-3acebfc6b762-istio-token" pod "362761bb-e08f-4109-8989-3acebfc6b762" (UID: "362761bb-e08f-4109-8989-3acebfc6b762"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:05:49.104964 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.104950    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/362761bb-e08f-4109-8989-3acebfc6b762-istio-podinfo" pod "362761bb-e08f-4109-8989-3acebfc6b762" (UID: "362761bb-e08f-4109-8989-3acebfc6b762"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:05:49.116497 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.116468    4926 generic.go:356] "Generic (PLEG): container finished" podID="362761bb-e08f-4109-8989-3acebfc6b762" containerID="45124883a5690f73fe168243314dfaf40f58ed2e28129b43f0d165e7c8835870" exitCode=0
Oct 31 04:05:49.116618 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.116513    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"
Oct 31 04:05:49.116618 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.116542    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv" event={"ID":"362761bb-e08f-4109-8989-3acebfc6b762","Type":"ContainerDied","Data":"45124883a5690f73fe168243314dfaf40f58ed2e28129b43f0d165e7c8835870"}
Oct 31 04:05:49.116618 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.116609    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv" event={"ID":"362761bb-e08f-4109-8989-3acebfc6b762","Type":"ContainerDied","Data":"a2f67786d1249e92a49a4ac9d7544514abbdb75a356238b11219c384d2beacef"}
Oct 31 04:05:49.116780 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.116637    4926 scope.go:122] "RemoveContainer" containerID="45124883a5690f73fe168243314dfaf40f58ed2e28129b43f0d165e7c8835870"
Oct 31 04:05:49.117982 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.117959    4926 generic.go:356] "Generic (PLEG): container finished" podID="88086a57-3749-41e6-8d6b-0fe95866237e" containerID="76f8e9a679714ae42f643032203cf655542e664add8fb0b0bb434be47c9cc263" exitCode=0
Oct 31 04:05:49.118072 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.118029    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92" event={"ID":"88086a57-3749-41e6-8d6b-0fe95866237e","Type":"ContainerDied","Data":"76f8e9a679714ae42f643032203cf655542e664add8fb0b0bb434be47c9cc263"}
Oct 31 04:05:49.118115 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.118074    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92" event={"ID":"88086a57-3749-41e6-8d6b-0fe95866237e","Type":"ContainerDied","Data":"321002edb7e39a9d540c0cac3f2191e26794feae013061fe497e978639c494ff"}
Oct 31 04:05:49.118115 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.118096    4926 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="321002edb7e39a9d540c0cac3f2191e26794feae013061fe497e978639c494ff"
Oct 31 04:05:49.120097 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.120080    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 04:05:49.127652 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.127634    4926 scope.go:122] "RemoveContainer" containerID="45124883a5690f73fe168243314dfaf40f58ed2e28129b43f0d165e7c8835870"
Oct 31 04:05:49.127935 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:05:49.127914    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45124883a5690f73fe168243314dfaf40f58ed2e28129b43f0d165e7c8835870\": container with ID starting with 45124883a5690f73fe168243314dfaf40f58ed2e28129b43f0d165e7c8835870 not found: ID does not exist" containerID="45124883a5690f73fe168243314dfaf40f58ed2e28129b43f0d165e7c8835870"
Oct 31 04:05:49.128016 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.127944    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45124883a5690f73fe168243314dfaf40f58ed2e28129b43f0d165e7c8835870"} err="failed to get container status \"45124883a5690f73fe168243314dfaf40f58ed2e28129b43f0d165e7c8835870\": rpc error: code = NotFound desc = could not find container \"45124883a5690f73fe168243314dfaf40f58ed2e28129b43f0d165e7c8835870\": container with ID starting with 45124883a5690f73fe168243314dfaf40f58ed2e28129b43f0d165e7c8835870 not found: ID does not exist"
Oct 31 04:05:49.203980 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.203947    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-workload-socket\") pod \"88086a57-3749-41e6-8d6b-0fe95866237e\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") "
Oct 31 04:05:49.204161 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.203990    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-istio-envoy\") pod \"88086a57-3749-41e6-8d6b-0fe95866237e\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") "
Oct 31 04:05:49.204161 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.204091    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88086a57-3749-41e6-8d6b-0fe95866237e-istio-token\" (UniqueName: \"kubernetes.io/projected/88086a57-3749-41e6-8d6b-0fe95866237e-istio-token\") pod \"88086a57-3749-41e6-8d6b-0fe95866237e\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") "
Oct 31 04:05:49.204161 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.204116    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/88086a57-3749-41e6-8d6b-0fe95866237e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88086a57-3749-41e6-8d6b-0fe95866237e-istiod-ca-cert\") pod \"88086a57-3749-41e6-8d6b-0fe95866237e\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") "
Oct 31 04:05:49.204366 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.204214    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-credential-socket\") pod \"88086a57-3749-41e6-8d6b-0fe95866237e\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") "
Oct 31 04:05:49.204366 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.204250    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-istio-data\") pod \"88086a57-3749-41e6-8d6b-0fe95866237e\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") "
Oct 31 04:05:49.204366 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.204250    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-workload-socket" pod "88086a57-3749-41e6-8d6b-0fe95866237e" (UID: "88086a57-3749-41e6-8d6b-0fe95866237e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:49.204366 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.204336    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-workload-certs\") pod \"88086a57-3749-41e6-8d6b-0fe95866237e\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") "
Oct 31 04:05:49.204572 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.204400    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/88086a57-3749-41e6-8d6b-0fe95866237e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88086a57-3749-41e6-8d6b-0fe95866237e-istio-podinfo\") pod \"88086a57-3749-41e6-8d6b-0fe95866237e\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") "
Oct 31 04:05:49.204572 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.204434    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88086a57-3749-41e6-8d6b-0fe95866237e-kube-api-access-z45q5\" (UniqueName: \"kubernetes.io/projected/88086a57-3749-41e6-8d6b-0fe95866237e-kube-api-access-z45q5\") pod \"88086a57-3749-41e6-8d6b-0fe95866237e\" (UID: \"88086a57-3749-41e6-8d6b-0fe95866237e\") "
Oct 31 04:05:49.204572 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.204565    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-istio-data" pod "88086a57-3749-41e6-8d6b-0fe95866237e" (UID: "88086a57-3749-41e6-8d6b-0fe95866237e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:49.204728 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.204590    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88086a57-3749-41e6-8d6b-0fe95866237e-istiod-ca-cert" pod "88086a57-3749-41e6-8d6b-0fe95866237e" (UID: "88086a57-3749-41e6-8d6b-0fe95866237e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:05:49.204776 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.204741    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-credential-socket" pod "88086a57-3749-41e6-8d6b-0fe95866237e" (UID: "88086a57-3749-41e6-8d6b-0fe95866237e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:49.204832 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.204804    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-workload-certs" pod "88086a57-3749-41e6-8d6b-0fe95866237e" (UID: "88086a57-3749-41e6-8d6b-0fe95866237e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:49.205141 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.205116    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.205300 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.205196    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88086a57-3749-41e6-8d6b-0fe95866237e-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.205394 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.205320    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/362761bb-e08f-4109-8989-3acebfc6b762-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.205394 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.205352    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qt6sm\" (UniqueName: \"kubernetes.io/projected/362761bb-e08f-4109-8989-3acebfc6b762-kube-api-access-qt6sm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.205394 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.205368    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/362761bb-e08f-4109-8989-3acebfc6b762-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.205394 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.205380    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.205645 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.205396    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.205645 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.205411    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.205645 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.205424    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/362761bb-e08f-4109-8989-3acebfc6b762-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.207421 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.207388    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88086a57-3749-41e6-8d6b-0fe95866237e-istio-token" pod "88086a57-3749-41e6-8d6b-0fe95866237e" (UID: "88086a57-3749-41e6-8d6b-0fe95866237e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:05:49.207505 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.207435    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-istio-envoy" pod "88086a57-3749-41e6-8d6b-0fe95866237e" (UID: "88086a57-3749-41e6-8d6b-0fe95866237e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:05:49.207590 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.207574    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/88086a57-3749-41e6-8d6b-0fe95866237e-istio-podinfo" pod "88086a57-3749-41e6-8d6b-0fe95866237e" (UID: "88086a57-3749-41e6-8d6b-0fe95866237e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:05:49.207782 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.207761    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88086a57-3749-41e6-8d6b-0fe95866237e-kube-api-access-z45q5" pod "88086a57-3749-41e6-8d6b-0fe95866237e" (UID: "88086a57-3749-41e6-8d6b-0fe95866237e"). InnerVolumeSpecName "kube-api-access-z45q5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:05:49.306556 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.306475    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88086a57-3749-41e6-8d6b-0fe95866237e-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.306556 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.306506    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88086a57-3749-41e6-8d6b-0fe95866237e-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.306556 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.306520    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88086a57-3749-41e6-8d6b-0fe95866237e-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.306556 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.306534    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z45q5\" (UniqueName: \"kubernetes.io/projected/88086a57-3749-41e6-8d6b-0fe95866237e-kube-api-access-z45q5\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:49.426075 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.426010    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"]
Oct 31 04:05:49.428590 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.428572    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-2lvsv"]
Oct 31 04:05:49.546592 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:49.546558    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="362761bb-e08f-4109-8989-3acebfc6b762" path="/var/lib/kubelet/pods/362761bb-e08f-4109-8989-3acebfc6b762/volumes"
Oct 31 04:05:50.125157 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:50.125086    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"
Oct 31 04:05:50.295597 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:50.295564    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"]
Oct 31 04:05:50.346805 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:50.346729    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-jmh92"]
Oct 31 04:05:51.539840 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:51.539807    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="88086a57-3749-41e6-8d6b-0fe95866237e" path="/var/lib/kubelet/pods/88086a57-3749-41e6-8d6b-0fe95866237e/volumes"
Oct 31 04:05:59.058344 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.058310    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"]
Oct 31 04:05:59.074204 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.059152    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp" podUID="9bdad2ce-6f22-4a30-b7e2-328bededd05b" containerName="istio-proxy" containerID="cri-o://3da33df7600481fecb67ea416981acd962ee48a92e6d881b8bff17e12d24aeae" gracePeriod=30
Oct 31 04:05:59.094904 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.094868    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"]
Oct 31 04:05:59.095377 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.095332    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6" podUID="bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" containerName="istio-proxy" containerID="cri-o://b035659f10ed9110fc2c2b692f4c176aa9c414be984805ed15d93e352ab5b90b" gracePeriod=30
Oct 31 04:05:59.185305 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.184684    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"]
Oct 31 04:05:59.185305 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.184923    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn" podUID="2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed" containerName="maas-api" containerID="cri-o://4d6c70c0c662671945f17d5e1ed82fea5a558d0d3d30480164ee0dbc0e8189fb" gracePeriod=30
Oct 31 04:05:59.608056 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.608022    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 04:05:59.714647 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.714614    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-maas-api-tls\") pod \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") "
Oct 31 04:05:59.714856 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.714733    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-kube-api-access-4q5qn\" (UniqueName: \"kubernetes.io/projected/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-kube-api-access-4q5qn\") pod \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") "
Oct 31 04:05:59.714856 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.714792    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-metrics-certs\") pod \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") "
Oct 31 04:05:59.714856 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.714835    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-openshift-service-ca\") pod \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\" (UID: \"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed\") "
Oct 31 04:05:59.715256 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.715226    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-openshift-service-ca" pod "2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed" (UID: "2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:05:59.715583 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.715442    4926 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-openshift-service-ca\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:59.716875 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.716850    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-maas-api-tls" pod "2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed" (UID: "2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:05:59.716991 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.716927    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-metrics-certs" pod "2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed" (UID: "2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:05:59.717253 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.717236    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-kube-api-access-4q5qn" pod "2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed" (UID: "2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed"). InnerVolumeSpecName "kube-api-access-4q5qn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:05:59.816644 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.816602    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q5qn\" (UniqueName: \"kubernetes.io/projected/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-kube-api-access-4q5qn\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:59.816644 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.816645    4926 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-metrics-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:05:59.816941 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:05:59.816661    4926 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed-maas-api-tls\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:00.214937 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:00.214905    4926 generic.go:356] "Generic (PLEG): container finished" podID="2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed" containerID="4d6c70c0c662671945f17d5e1ed82fea5a558d0d3d30480164ee0dbc0e8189fb" exitCode=0
Oct 31 04:06:00.215481 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:00.214951    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn" event={"ID":"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed","Type":"ContainerDied","Data":"4d6c70c0c662671945f17d5e1ed82fea5a558d0d3d30480164ee0dbc0e8189fb"}
Oct 31 04:06:00.215481 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:00.214958    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"
Oct 31 04:06:00.215481 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:00.214995    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn" event={"ID":"2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed","Type":"ContainerDied","Data":"bba784583d2d232c9805faaf4175346d2eb4e0ef48e6b083605620d44a17d61b"}
Oct 31 04:06:00.215481 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:00.215016    4926 scope.go:122] "RemoveContainer" containerID="4d6c70c0c662671945f17d5e1ed82fea5a558d0d3d30480164ee0dbc0e8189fb"
Oct 31 04:06:00.228475 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:00.228446    4926 scope.go:122] "RemoveContainer" containerID="4d6c70c0c662671945f17d5e1ed82fea5a558d0d3d30480164ee0dbc0e8189fb"
Oct 31 04:06:00.228807 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:00.228785    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d6c70c0c662671945f17d5e1ed82fea5a558d0d3d30480164ee0dbc0e8189fb\": container with ID starting with 4d6c70c0c662671945f17d5e1ed82fea5a558d0d3d30480164ee0dbc0e8189fb not found: ID does not exist" containerID="4d6c70c0c662671945f17d5e1ed82fea5a558d0d3d30480164ee0dbc0e8189fb"
Oct 31 04:06:00.228875 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:00.228823    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d6c70c0c662671945f17d5e1ed82fea5a558d0d3d30480164ee0dbc0e8189fb"} err="failed to get container status \"4d6c70c0c662671945f17d5e1ed82fea5a558d0d3d30480164ee0dbc0e8189fb\": rpc error: code = NotFound desc = could not find container \"4d6c70c0c662671945f17d5e1ed82fea5a558d0d3d30480164ee0dbc0e8189fb\": container with ID starting with 4d6c70c0c662671945f17d5e1ed82fea5a558d0d3d30480164ee0dbc0e8189fb not found: ID does not exist"
Oct 31 04:06:00.395488 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:00.395452    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"]
Oct 31 04:06:00.447030 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:00.446996    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-68aa3e-7df694865-sxtzn"]
Oct 31 04:06:01.342053 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:01.342015    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:06:01.342453 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:01.342064    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp" podUID="9bdad2ce-6f22-4a30-b7e2-328bededd05b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:06:01.540165 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:01.540134    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed" path="/var/lib/kubelet/pods/2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed/volumes"
Oct 31 04:06:03.440565 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:03.440522    4926 patch_prober.go:29] interesting pod/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:06:03.440967 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:03.440607    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6" podUID="bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:06:04.243347 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.243307    4926 generic.go:356] "Generic (PLEG): container finished" podID="bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" containerID="b035659f10ed9110fc2c2b692f4c176aa9c414be984805ed15d93e352ab5b90b" exitCode=0
Oct 31 04:06:04.243513 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.243353    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6" event={"ID":"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3","Type":"ContainerDied","Data":"b035659f10ed9110fc2c2b692f4c176aa9c414be984805ed15d93e352ab5b90b"}
Oct 31 04:06:04.245218 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.245192    4926 generic.go:356] "Generic (PLEG): container finished" podID="9bdad2ce-6f22-4a30-b7e2-328bededd05b" containerID="3da33df7600481fecb67ea416981acd962ee48a92e6d881b8bff17e12d24aeae" exitCode=0
Oct 31 04:06:04.245349 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.245251    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp" event={"ID":"9bdad2ce-6f22-4a30-b7e2-328bededd05b","Type":"ContainerDied","Data":"3da33df7600481fecb67ea416981acd962ee48a92e6d881b8bff17e12d24aeae"}
Oct 31 04:06:04.372125 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.372061    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 04:06:04.426816 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.426788    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 04:06:04.466768 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.466741    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istiod-ca-cert\") pod \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") "
Oct 31 04:06:04.467169 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.466778    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-podinfo\") pod \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") "
Oct 31 04:06:04.467169 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.466838    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-workload-socket\") pod \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") "
Oct 31 04:06:04.467169 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.466880    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-credential-socket\") pod \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") "
Oct 31 04:06:04.467169 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.466916    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-workload-socket\") pod \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") "
Oct 31 04:06:04.467169 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.466951    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-envoy\") pod \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") "
Oct 31 04:06:04.467169 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467028    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-envoy\") pod \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") "
Oct 31 04:06:04.467169 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467067    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9bdad2ce-6f22-4a30-b7e2-328bededd05b-kube-api-access-kkcmz\" (UniqueName: \"kubernetes.io/projected/9bdad2ce-6f22-4a30-b7e2-328bededd05b-kube-api-access-kkcmz\") pod \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") "
Oct 31 04:06:04.467169 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467125    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-token\" (UniqueName: \"kubernetes.io/projected/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-token\") pod \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") "
Oct 31 04:06:04.467169 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467159    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-credential-socket" pod "9bdad2ce-6f22-4a30-b7e2-328bededd05b" (UID: "9bdad2ce-6f22-4a30-b7e2-328bededd05b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:04.467648 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467182    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-credential-socket\") pod \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") "
Oct 31 04:06:04.467648 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467187    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istiod-ca-cert" pod "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" (UID: "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:06:04.467648 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467209    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istiod-ca-cert\") pod \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") "
Oct 31 04:06:04.467648 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467236    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-token\" (UniqueName: \"kubernetes.io/projected/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-token\") pod \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") "
Oct 31 04:06:04.467648 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467308    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-workload-certs\") pod \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") "
Oct 31 04:06:04.467648 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467347    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-workload-socket" pod "9bdad2ce-6f22-4a30-b7e2-328bededd05b" (UID: "9bdad2ce-6f22-4a30-b7e2-328bededd05b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:04.467648 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467430    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-podinfo\") pod \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") "
Oct 31 04:06:04.467648 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467482    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-kube-api-access-72wh8\" (UniqueName: \"kubernetes.io/projected/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-kube-api-access-72wh8\") pod \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") "
Oct 31 04:06:04.467648 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467520    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-data\") pod \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") "
Oct 31 04:06:04.467648 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467565    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-data\") pod \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\" (UID: \"9bdad2ce-6f22-4a30-b7e2-328bededd05b\") "
Oct 31 04:06:04.467648 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467620    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-workload-certs\") pod \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\" (UID: \"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3\") "
Oct 31 04:06:04.468178 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467665    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istiod-ca-cert" pod "9bdad2ce-6f22-4a30-b7e2-328bededd05b" (UID: "9bdad2ce-6f22-4a30-b7e2-328bededd05b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:06:04.468178 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467770    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-workload-socket" pod "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" (UID: "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:04.468178 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.467931    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-credential-socket" pod "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" (UID: "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:04.468178 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.468159    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-workload-certs" pod "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" (UID: "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:04.468543 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.468522    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.468639 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.468603    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.468639 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.468622    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.468639 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.468636    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.468803 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.468649    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.468803 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.468663    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.468803 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.468678    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.468959 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.468890    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-data" pod "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" (UID: "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:04.469817 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.469790    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-data" pod "9bdad2ce-6f22-4a30-b7e2-328bededd05b" (UID: "9bdad2ce-6f22-4a30-b7e2-328bededd05b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:04.469942 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.469843    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-workload-certs" pod "9bdad2ce-6f22-4a30-b7e2-328bededd05b" (UID: "9bdad2ce-6f22-4a30-b7e2-328bededd05b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:04.470331 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.470288    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-envoy" pod "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" (UID: "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:04.470473 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.470450    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-podinfo" pod "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" (UID: "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:06:04.470589 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.470564    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-token" pod "9bdad2ce-6f22-4a30-b7e2-328bededd05b" (UID: "9bdad2ce-6f22-4a30-b7e2-328bededd05b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:04.470702 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.470665    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bdad2ce-6f22-4a30-b7e2-328bededd05b-kube-api-access-kkcmz" pod "9bdad2ce-6f22-4a30-b7e2-328bededd05b" (UID: "9bdad2ce-6f22-4a30-b7e2-328bededd05b"). InnerVolumeSpecName "kube-api-access-kkcmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:04.470767 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.470750    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-podinfo" pod "9bdad2ce-6f22-4a30-b7e2-328bededd05b" (UID: "9bdad2ce-6f22-4a30-b7e2-328bededd05b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:06:04.470804 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.470764    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-kube-api-access-72wh8" pod "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" (UID: "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3"). InnerVolumeSpecName "kube-api-access-72wh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:04.471471 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.471452    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-token" pod "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" (UID: "bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:04.471537 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.471463    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-envoy" pod "9bdad2ce-6f22-4a30-b7e2-328bededd05b" (UID: "9bdad2ce-6f22-4a30-b7e2-328bededd05b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:04.570503 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.569987    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.570503 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.570022    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.570503 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.570037    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.570503 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.570054    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.570503 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.570068    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.570503 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.570084    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkcmz\" (UniqueName: \"kubernetes.io/projected/9bdad2ce-6f22-4a30-b7e2-328bededd05b-kube-api-access-kkcmz\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.570503 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.570099    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.570503 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.570112    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.570503 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.570126    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bdad2ce-6f22-4a30-b7e2-328bededd05b-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.570503 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.570138    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bdad2ce-6f22-4a30-b7e2-328bededd05b-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:04.570503 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:04.570152    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72wh8\" (UniqueName: \"kubernetes.io/projected/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3-kube-api-access-72wh8\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:05.252803 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.252775    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"
Oct 31 04:06:05.252993 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.252808    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp" event={"ID":"9bdad2ce-6f22-4a30-b7e2-328bededd05b","Type":"ContainerDied","Data":"753ba049068f2eaab7aac077adc031957bdbbe2282571a4850596eebd3804c20"}
Oct 31 04:06:05.252993 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.252882    4926 scope.go:122] "RemoveContainer" containerID="3da33df7600481fecb67ea416981acd962ee48a92e6d881b8bff17e12d24aeae"
Oct 31 04:06:05.255669 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.255638    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6" event={"ID":"bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3","Type":"ContainerDied","Data":"b20455195a21b4c052c30eab49362146f11e9ddf2bc02432807ada5a1eae5315"}
Oct 31 04:06:05.255898 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.255796    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"
Oct 31 04:06:05.275004 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.274984    4926 scope.go:122] "RemoveContainer" containerID="b035659f10ed9110fc2c2b692f4c176aa9c414be984805ed15d93e352ab5b90b"
Oct 31 04:06:05.312154 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:05.312122    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:06:05.422915 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.422839    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"]
Oct 31 04:06:05.474528 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.474494    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-wpbl6"]
Oct 31 04:06:05.540692 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.540649    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" path="/var/lib/kubelet/pods/bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3/volumes"
Oct 31 04:06:05.540981 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.540967    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"]
Oct 31 04:06:05.543118 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.543091    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-8kfrp"]
Oct 31 04:06:05.683078 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683000    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"]
Oct 31 04:06:05.683472 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683460    4926 cpu_manager.go:414] "containerMap: removing container" podUID="2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed" containerName="maas-api"
Oct 31 04:06:05.683524 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683474    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed" containerName="maas-api"
Oct 31 04:06:05.683524 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683486    4926 cpu_manager.go:414] "containerMap: removing container" podUID="88086a57-3749-41e6-8d6b-0fe95866237e" containerName="istio-proxy"
Oct 31 04:06:05.683524 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683491    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88086a57-3749-41e6-8d6b-0fe95866237e" containerName="istio-proxy"
Oct 31 04:06:05.683524 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683501    4926 cpu_manager.go:414] "containerMap: removing container" podUID="7d0301a0-5d7b-4305-b4dd-24539a745220" containerName="discovery"
Oct 31 04:06:05.683524 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683506    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d0301a0-5d7b-4305-b4dd-24539a745220" containerName="discovery"
Oct 31 04:06:05.683524 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683521    4926 cpu_manager.go:414] "containerMap: removing container" podUID="9bdad2ce-6f22-4a30-b7e2-328bededd05b" containerName="istio-proxy"
Oct 31 04:06:05.683524 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683525    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9bdad2ce-6f22-4a30-b7e2-328bededd05b" containerName="istio-proxy"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683540    4926 cpu_manager.go:414] "containerMap: removing container" podUID="2524a945-cbb0-42b3-b2be-8cd43a1f1df9" containerName="istio-proxy"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683545    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2524a945-cbb0-42b3-b2be-8cd43a1f1df9" containerName="istio-proxy"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683551    4926 cpu_manager.go:414] "containerMap: removing container" podUID="362761bb-e08f-4109-8989-3acebfc6b762" containerName="istio-proxy"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683555    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="362761bb-e08f-4109-8989-3acebfc6b762" containerName="istio-proxy"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683563    4926 cpu_manager.go:414] "containerMap: removing container" podUID="08979c91-eaa0-4650-ace2-eda96f061f1d" containerName="istio-proxy"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683567    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08979c91-eaa0-4650-ace2-eda96f061f1d" containerName="istio-proxy"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683582    4926 cpu_manager.go:414] "containerMap: removing container" podUID="08979c91-eaa0-4650-ace2-eda96f061f1d" containerName="istio-proxy"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683587    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08979c91-eaa0-4650-ace2-eda96f061f1d" containerName="istio-proxy"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683597    4926 cpu_manager.go:414] "containerMap: removing container" podUID="766c0067-9aba-4388-906d-171ff0dd333c" containerName="cleanup"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683600    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="766c0067-9aba-4388-906d-171ff0dd333c" containerName="cleanup"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683608    4926 cpu_manager.go:414] "containerMap: removing container" podUID="bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" containerName="istio-proxy"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683612    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" containerName="istio-proxy"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683619    4926 cpu_manager.go:414] "containerMap: removing container" podUID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf" containerName="cleanup"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683624    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf" containerName="cleanup"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683704    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc4ddab3-11f4-42d1-818a-8e1cf7fd34e3" containerName="istio-proxy"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683713    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bbdcaf9-4e87-408d-8ea2-8a4be2beefcf" containerName="cleanup"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683721    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c5bc95d-4ff6-4049-8e4c-ec0725ca68ed" containerName="maas-api"
Oct 31 04:06:05.683722 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683727    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="88086a57-3749-41e6-8d6b-0fe95866237e" containerName="istio-proxy"
Oct 31 04:06:05.684203 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683736    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d0301a0-5d7b-4305-b4dd-24539a745220" containerName="discovery"
Oct 31 04:06:05.684203 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683743    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="9bdad2ce-6f22-4a30-b7e2-328bededd05b" containerName="istio-proxy"
Oct 31 04:06:05.684203 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683752    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="2524a945-cbb0-42b3-b2be-8cd43a1f1df9" containerName="istio-proxy"
Oct 31 04:06:05.684203 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683757    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="9dec67dd-f744-4a84-8c94-10e91694a5d5" containerName="cleanup"
Oct 31 04:06:05.684203 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683763    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="362761bb-e08f-4109-8989-3acebfc6b762" containerName="istio-proxy"
Oct 31 04:06:05.684203 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683767    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="08979c91-eaa0-4650-ace2-eda96f061f1d" containerName="istio-proxy"
Oct 31 04:06:05.684203 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.683774    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="08979c91-eaa0-4650-ace2-eda96f061f1d" containerName="istio-proxy"
Oct 31 04:06:05.688635 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.688620    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:05.741723 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.741696    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"]
Oct 31 04:06:05.742646 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.742619    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-3473a4-serving-cert\""
Oct 31 04:06:05.783604 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.783577    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-metrics-certs\") pod \"maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:05.783756 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.783615    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07d7017-aef8-49fb-8de0-0b3cba42c240-openshift-service-ca\") pod \"maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:05.783832 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.783812    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-maas-api-tls\") pod \"maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:05.783892 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.783871    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmwg6\" (UniqueName: \"kubernetes.io/projected/a07d7017-aef8-49fb-8de0-0b3cba42c240-kube-api-access-lmwg6\") pod \"maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:05.884689 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.884657    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-metrics-certs\") pod \"maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:05.884869 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.884702    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07d7017-aef8-49fb-8de0-0b3cba42c240-openshift-service-ca\") pod \"maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:05.884869 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.884817    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-maas-api-tls\") pod \"maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:05.884869 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.884849    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmwg6\" (UniqueName: \"kubernetes.io/projected/a07d7017-aef8-49fb-8de0-0b3cba42c240-kube-api-access-lmwg6\") pod \"maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:05.884998 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:05.884948    4926 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-serving-cert: secret "maas-api-e2e-api-b-3473a4-serving-cert" not found
Oct 31 04:06:05.885032 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:05.885012    4926 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-maas-api-tls podName:a07d7017-aef8-49fb-8de0-0b3cba42c240 nodeName:}" failed. No retries permitted until 2026-10-31 04:06:06.384996159 +0000 UTC m=+2262.970536419 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-maas-api-tls") pod "maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv" (UID: "a07d7017-aef8-49fb-8de0-0b3cba42c240") : secret "maas-api-e2e-api-b-3473a4-serving-cert" not found
Oct 31 04:06:05.885459 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.885436    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07d7017-aef8-49fb-8de0-0b3cba42c240-openshift-service-ca\") pod \"maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:05.887407 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.887383    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-metrics-certs\") pod \"maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:05.948694 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:05.948617    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmwg6\" (UniqueName: \"kubernetes.io/projected/a07d7017-aef8-49fb-8de0-0b3cba42c240-kube-api-access-lmwg6\") pod \"maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:06.389951 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:06.389913    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-maas-api-tls\") pod \"maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:06.392599 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:06.392574    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-maas-api-tls\") pod \"maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:06.602141 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:06.602108    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:06.971874 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:06:06.971840    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda07d7017_aef8_49fb_8de0_0b3cba42c240.slice/crio-3feb40cc56585804a33198ee0a2eb642112383820af4660baa56c52331af193e WatchSource:0}: Error finding container 3feb40cc56585804a33198ee0a2eb642112383820af4660baa56c52331af193e: Status 404 returned error can't find the container with id 3feb40cc56585804a33198ee0a2eb642112383820af4660baa56c52331af193e
Oct 31 04:06:06.973510 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:06.973484    4926 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 04:06:06.975896 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:06.975870    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"]
Oct 31 04:06:07.271281 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:07.271177    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv" event={"ID":"a07d7017-aef8-49fb-8de0-0b3cba42c240","Type":"ContainerStarted","Data":"3feb40cc56585804a33198ee0a2eb642112383820af4660baa56c52331af193e"}
Oct 31 04:06:07.539471 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:07.539387    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9bdad2ce-6f22-4a30-b7e2-328bededd05b" path="/var/lib/kubelet/pods/9bdad2ce-6f22-4a30-b7e2-328bededd05b/volumes"
Oct 31 04:06:08.141236 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:08.141154    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"]
Oct 31 04:06:08.141715 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:08.141358    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj" podUID="d5125225-3fc9-4a55-8df9-0bce428ea38d" containerName="istio-proxy" containerID="cri-o://87dcb492a8877a5ca035962fbc223c69bf33e58110e90ee2ce3fc5e107fa5245" gracePeriod=30
Oct 31 04:06:08.283785 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:08.283747    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv" event={"ID":"a07d7017-aef8-49fb-8de0-0b3cba42c240","Type":"ContainerStarted","Data":"8ef6ee32cff2ffa1f047256fc76443df5b460280cefc7eb07207d45b8fe32aa2"}
Oct 31 04:06:08.284704 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:08.284660    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:08.406687 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:08.406578    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv" podStartSLOduration=2.489701992 podStartE2EDuration="3.406563928s" totalImagesPullingTime="916.861936ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:06:08.406056069 +0000 UTC m=+2264.991596330" watchObservedRunningTime="2026-10-31 04:06:08.406563928 +0000 UTC m=+2264.992104229"
Oct 31 04:06:09.661090 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:09.661058    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6950-7a7bf7"]
Oct 31 04:06:09.665235 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:09.665212    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-7a7bf7"
Oct 31 04:06:09.726422 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:09.723660    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6950-7a7bf7"]
Oct 31 04:06:09.732692 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:09.732664    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npk4x\" (UniqueName: \"kubernetes.io/projected/a9bd409b-1b3f-4238-a2b6-4d9a5925d728-kube-api-access-npk4x\") pod \"test-curl-6950-7a7bf7\" (UID: \"a9bd409b-1b3f-4238-a2b6-4d9a5925d728\") " pod="opendatahub/test-curl-6950-7a7bf7"
Oct 31 04:06:09.772479 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:09.772442    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-2x2q5\""
Oct 31 04:06:09.833675 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:09.833642    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npk4x\" (UniqueName: \"kubernetes.io/projected/a9bd409b-1b3f-4238-a2b6-4d9a5925d728-kube-api-access-npk4x\") pod \"test-curl-6950-7a7bf7\" (UID: \"a9bd409b-1b3f-4238-a2b6-4d9a5925d728\") " pod="opendatahub/test-curl-6950-7a7bf7"
Oct 31 04:06:09.896060 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:09.896021    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npk4x\" (UniqueName: \"kubernetes.io/projected/a9bd409b-1b3f-4238-a2b6-4d9a5925d728-kube-api-access-npk4x\") pod \"test-curl-6950-7a7bf7\" (UID: \"a9bd409b-1b3f-4238-a2b6-4d9a5925d728\") " pod="opendatahub/test-curl-6950-7a7bf7"
Oct 31 04:06:09.978546 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:09.978515    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-7a7bf7"
Oct 31 04:06:10.372516 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:06:10.372481    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9bd409b_1b3f_4238_a2b6_4d9a5925d728.slice/crio-d2b36a9426584776167df1e42fcc0443a47394185bd50c24baff670a5ceb02e5 WatchSource:0}: Error finding container d2b36a9426584776167df1e42fcc0443a47394185bd50c24baff670a5ceb02e5: Status 404 returned error can't find the container with id d2b36a9426584776167df1e42fcc0443a47394185bd50c24baff670a5ceb02e5
Oct 31 04:06:10.375466 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:10.375424    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6950-7a7bf7"]
Oct 31 04:06:11.301823 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:11.301791    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6950-7a7bf7" event={"ID":"a9bd409b-1b3f-4238-a2b6-4d9a5925d728","Type":"ContainerStarted","Data":"a874f6a00f9473307295e83c7f0567e3f7305a631b27d2cb00ec4f4ea856f28a"}
Oct 31 04:06:11.302216 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:11.301836    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6950-7a7bf7" event={"ID":"a9bd409b-1b3f-4238-a2b6-4d9a5925d728","Type":"ContainerStarted","Data":"d2b36a9426584776167df1e42fcc0443a47394185bd50c24baff670a5ceb02e5"}
Oct 31 04:06:11.420099 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:11.420051    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6950-7a7bf7" podStartSLOduration=2.42003776 podStartE2EDuration="2.42003776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:06:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:06:11.418456544 +0000 UTC m=+2268.003996808" watchObservedRunningTime="2026-10-31 04:06:11.42003776 +0000 UTC m=+2268.005578021"
Oct 31 04:06:12.903170 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:12.903138    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6950-7a7bf7"]
Oct 31 04:06:12.907697 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:12.907668    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6950-7a7bf7"]
Oct 31 04:06:13.070408 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.070377    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6950-387ea4"]
Oct 31 04:06:13.070882 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.070869    4926 cpu_manager.go:414] "containerMap: removing container" podUID="a9bd409b-1b3f-4238-a2b6-4d9a5925d728" containerName="test-curl-6950-7a7bf7"
Oct 31 04:06:13.070948 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.070884    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9bd409b-1b3f-4238-a2b6-4d9a5925d728" containerName="test-curl-6950-7a7bf7"
Oct 31 04:06:13.070984 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.070961    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9bd409b-1b3f-4238-a2b6-4d9a5925d728" containerName="test-curl-6950-7a7bf7"
Oct 31 04:06:13.074214 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.074197    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-387ea4"
Oct 31 04:06:13.147673 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.147439    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6950-387ea4"]
Oct 31 04:06:13.182036 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.182009    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhl5k\" (UniqueName: \"kubernetes.io/projected/7cfff2c4-773e-4523-b442-3b388f1f6894-kube-api-access-rhl5k\") pod \"test-curl-6950-387ea4\" (UID: \"7cfff2c4-773e-4523-b442-3b388f1f6894\") " pod="opendatahub/test-curl-6950-387ea4"
Oct 31 04:06:13.283028 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.282989    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhl5k\" (UniqueName: \"kubernetes.io/projected/7cfff2c4-773e-4523-b442-3b388f1f6894-kube-api-access-rhl5k\") pod \"test-curl-6950-387ea4\" (UID: \"7cfff2c4-773e-4523-b442-3b388f1f6894\") " pod="opendatahub/test-curl-6950-387ea4"
Oct 31 04:06:13.315608 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.315578    4926 generic.go:356] "Generic (PLEG): container finished" podID="d5125225-3fc9-4a55-8df9-0bce428ea38d" containerID="87dcb492a8877a5ca035962fbc223c69bf33e58110e90ee2ce3fc5e107fa5245" exitCode=0
Oct 31 04:06:13.316420 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.315760    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj" event={"ID":"d5125225-3fc9-4a55-8df9-0bce428ea38d","Type":"ContainerDied","Data":"87dcb492a8877a5ca035962fbc223c69bf33e58110e90ee2ce3fc5e107fa5245"}
Oct 31 04:06:13.316420 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.315835    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6950-7a7bf7" podUID="a9bd409b-1b3f-4238-a2b6-4d9a5925d728" containerName="test-curl-6950-7a7bf7" containerID="cri-o://a874f6a00f9473307295e83c7f0567e3f7305a631b27d2cb00ec4f4ea856f28a" gracePeriod=2
Oct 31 04:06:13.347788 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.347758    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhl5k\" (UniqueName: \"kubernetes.io/projected/7cfff2c4-773e-4523-b442-3b388f1f6894-kube-api-access-rhl5k\") pod \"test-curl-6950-387ea4\" (UID: \"7cfff2c4-773e-4523-b442-3b388f1f6894\") " pod="opendatahub/test-curl-6950-387ea4"
Oct 31 04:06:13.368807 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:13.368766    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-7a7bf7\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="a9bd409b-1b3f-4238-a2b6-4d9a5925d728" pod="opendatahub/test-curl-6950-7a7bf7"
Oct 31 04:06:13.386671 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.386646    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-387ea4"
Oct 31 04:06:13.423687 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.423625    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 04:06:13.484990 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.484960    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-workload-socket\") pod \"d5125225-3fc9-4a55-8df9-0bce428ea38d\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") "
Oct 31 04:06:13.484990 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.484993    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-credential-socket\") pod \"d5125225-3fc9-4a55-8df9-0bce428ea38d\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") "
Oct 31 04:06:13.485206 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.485018    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-data\") pod \"d5125225-3fc9-4a55-8df9-0bce428ea38d\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") "
Oct 31 04:06:13.485206 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.485083    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-envoy\") pod \"d5125225-3fc9-4a55-8df9-0bce428ea38d\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") "
Oct 31 04:06:13.485206 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.485178    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-workload-certs\") pod \"d5125225-3fc9-4a55-8df9-0bce428ea38d\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") "
Oct 31 04:06:13.485385 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.485334    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-token\" (UniqueName: \"kubernetes.io/projected/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-token\") pod \"d5125225-3fc9-4a55-8df9-0bce428ea38d\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") "
Oct 31 04:06:13.485446 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.485422    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5125225-3fc9-4a55-8df9-0bce428ea38d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5125225-3fc9-4a55-8df9-0bce428ea38d-istiod-ca-cert\") pod \"d5125225-3fc9-4a55-8df9-0bce428ea38d\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") "
Oct 31 04:06:13.485509 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.485445    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-data" pod "d5125225-3fc9-4a55-8df9-0bce428ea38d" (UID: "d5125225-3fc9-4a55-8df9-0bce428ea38d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:13.485509 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.485497    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-podinfo\") pod \"d5125225-3fc9-4a55-8df9-0bce428ea38d\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") "
Oct 31 04:06:13.485609 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.485562    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5125225-3fc9-4a55-8df9-0bce428ea38d-kube-api-access-9crl7\" (UniqueName: \"kubernetes.io/projected/d5125225-3fc9-4a55-8df9-0bce428ea38d-kube-api-access-9crl7\") pod \"d5125225-3fc9-4a55-8df9-0bce428ea38d\" (UID: \"d5125225-3fc9-4a55-8df9-0bce428ea38d\") "
Oct 31 04:06:13.486138 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.485564    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-workload-socket" pod "d5125225-3fc9-4a55-8df9-0bce428ea38d" (UID: "d5125225-3fc9-4a55-8df9-0bce428ea38d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:13.486138 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.485715    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-workload-certs" pod "d5125225-3fc9-4a55-8df9-0bce428ea38d" (UID: "d5125225-3fc9-4a55-8df9-0bce428ea38d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:13.486138 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.485999    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-credential-socket" pod "d5125225-3fc9-4a55-8df9-0bce428ea38d" (UID: "d5125225-3fc9-4a55-8df9-0bce428ea38d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:13.486138 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.486041    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5125225-3fc9-4a55-8df9-0bce428ea38d-istiod-ca-cert" pod "d5125225-3fc9-4a55-8df9-0bce428ea38d" (UID: "d5125225-3fc9-4a55-8df9-0bce428ea38d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:06:13.486461 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.486171    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:13.488488 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.488454    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-token" pod "d5125225-3fc9-4a55-8df9-0bce428ea38d" (UID: "d5125225-3fc9-4a55-8df9-0bce428ea38d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:13.488685 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.488657    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-podinfo" pod "d5125225-3fc9-4a55-8df9-0bce428ea38d" (UID: "d5125225-3fc9-4a55-8df9-0bce428ea38d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:06:13.489129 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.489107    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5125225-3fc9-4a55-8df9-0bce428ea38d-kube-api-access-9crl7" pod "d5125225-3fc9-4a55-8df9-0bce428ea38d" (UID: "d5125225-3fc9-4a55-8df9-0bce428ea38d"). InnerVolumeSpecName "kube-api-access-9crl7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:13.489515 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.489489    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-envoy" pod "d5125225-3fc9-4a55-8df9-0bce428ea38d" (UID: "d5125225-3fc9-4a55-8df9-0bce428ea38d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:13.587091 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.587044    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:13.587091 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.587088    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9crl7\" (UniqueName: \"kubernetes.io/projected/d5125225-3fc9-4a55-8df9-0bce428ea38d-kube-api-access-9crl7\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:13.587091 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.587100    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:13.587091 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.587109    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:13.587378 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.587118    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:13.587378 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.587126    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5125225-3fc9-4a55-8df9-0bce428ea38d-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:13.587378 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.587133    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5125225-3fc9-4a55-8df9-0bce428ea38d-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:13.587378 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.587141    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5125225-3fc9-4a55-8df9-0bce428ea38d-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:13.599201 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:13.599165    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-7a7bf7\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="a9bd409b-1b3f-4238-a2b6-4d9a5925d728" pod="opendatahub/test-curl-6950-7a7bf7"
Oct 31 04:06:13.757798 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:06:13.757755    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7cfff2c4_773e_4523_b442_3b388f1f6894.slice/crio-02e3f3946c26642f56bf2ad183103b4f89cf1992b83133b9bc56f88a2cd851e8 WatchSource:0}: Error finding container 02e3f3946c26642f56bf2ad183103b4f89cf1992b83133b9bc56f88a2cd851e8: Status 404 returned error can't find the container with id 02e3f3946c26642f56bf2ad183103b4f89cf1992b83133b9bc56f88a2cd851e8
Oct 31 04:06:13.768318 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:13.768284    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6950-387ea4"]
Oct 31 04:06:14.324238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:14.324200    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj" event={"ID":"d5125225-3fc9-4a55-8df9-0bce428ea38d","Type":"ContainerDied","Data":"a83dedc60e0769728d674742eb5c3fbe56190528fc5827c2be461e68808c4d3f"}
Oct 31 04:06:14.324238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:14.324221    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"
Oct 31 04:06:14.324740 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:14.324306    4926 scope.go:122] "RemoveContainer" containerID="87dcb492a8877a5ca035962fbc223c69bf33e58110e90ee2ce3fc5e107fa5245"
Oct 31 04:06:14.325994 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:14.325959    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6950-387ea4" event={"ID":"7cfff2c4-773e-4523-b442-3b388f1f6894","Type":"ContainerStarted","Data":"6b214311329a7712c427d8b8ab315fe2c3331efc72df4c308c41855cfc41a429"}
Oct 31 04:06:14.325994 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:14.325994    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6950-387ea4" event={"ID":"7cfff2c4-773e-4523-b442-3b388f1f6894","Type":"ContainerStarted","Data":"02e3f3946c26642f56bf2ad183103b4f89cf1992b83133b9bc56f88a2cd851e8"}
Oct 31 04:06:14.494820 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:14.494788    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"]
Oct 31 04:06:14.546065 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:14.546032    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-68aa3e-openshift-default-5f5c8bdd5-klksj"]
Oct 31 04:06:14.615307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:14.615201    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6950-387ea4" podStartSLOduration=1.615188727 podStartE2EDuration="1.615188727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:06:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:06:14.614163103 +0000 UTC m=+2271.199703362" watchObservedRunningTime="2026-10-31 04:06:14.615188727 +0000 UTC m=+2271.200728989"
Oct 31 04:06:15.300548 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:15.300522    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:06:15.565552 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:15.565466    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5125225-3fc9-4a55-8df9-0bce428ea38d" path="/var/lib/kubelet/pods/d5125225-3fc9-4a55-8df9-0bce428ea38d/volumes"
Oct 31 04:06:15.597827 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:15.597801    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-7a7bf7"
Oct 31 04:06:15.650534 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:15.650498    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-7a7bf7\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="a9bd409b-1b3f-4238-a2b6-4d9a5925d728" pod="opendatahub/test-curl-6950-7a7bf7"
Oct 31 04:06:15.715222 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:15.715195    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9bd409b-1b3f-4238-a2b6-4d9a5925d728-kube-api-access-npk4x\" (UniqueName: \"kubernetes.io/projected/a9bd409b-1b3f-4238-a2b6-4d9a5925d728-kube-api-access-npk4x\") pod \"a9bd409b-1b3f-4238-a2b6-4d9a5925d728\" (UID: \"a9bd409b-1b3f-4238-a2b6-4d9a5925d728\") "
Oct 31 04:06:15.717475 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:15.717451    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9bd409b-1b3f-4238-a2b6-4d9a5925d728-kube-api-access-npk4x" pod "a9bd409b-1b3f-4238-a2b6-4d9a5925d728" (UID: "a9bd409b-1b3f-4238-a2b6-4d9a5925d728"). InnerVolumeSpecName "kube-api-access-npk4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:15.816532 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:15.816464    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npk4x\" (UniqueName: \"kubernetes.io/projected/a9bd409b-1b3f-4238-a2b6-4d9a5925d728-kube-api-access-npk4x\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:16.100789 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.100705    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6950-387ea4"]
Oct 31 04:06:16.104126 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.104094    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6950-387ea4"]
Oct 31 04:06:16.257676 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.257643    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6950-1c122a"]
Oct 31 04:06:16.258151 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.258136    4926 cpu_manager.go:414] "containerMap: removing container" podUID="d5125225-3fc9-4a55-8df9-0bce428ea38d" containerName="istio-proxy"
Oct 31 04:06:16.258193 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.258153    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5125225-3fc9-4a55-8df9-0bce428ea38d" containerName="istio-proxy"
Oct 31 04:06:16.258193 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.258170    4926 cpu_manager.go:414] "containerMap: removing container" podUID="7cfff2c4-773e-4523-b442-3b388f1f6894" containerName="test-curl-6950-387ea4"
Oct 31 04:06:16.258193 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.258176    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7cfff2c4-773e-4523-b442-3b388f1f6894" containerName="test-curl-6950-387ea4"
Oct 31 04:06:16.258319 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.258253    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5125225-3fc9-4a55-8df9-0bce428ea38d" containerName="istio-proxy"
Oct 31 04:06:16.258319 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.258262    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="7cfff2c4-773e-4523-b442-3b388f1f6894" containerName="test-curl-6950-387ea4"
Oct 31 04:06:16.261461 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.261446    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-1c122a"
Oct 31 04:06:16.313949 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.313921    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6950-1c122a"]
Oct 31 04:06:16.322900 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.322877    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpdk8\" (UniqueName: \"kubernetes.io/projected/81d0bdd5-01cb-4617-bca8-1928c08ff9fc-kube-api-access-cpdk8\") pod \"test-curl-6950-1c122a\" (UID: \"81d0bdd5-01cb-4617-bca8-1928c08ff9fc\") " pod="opendatahub/test-curl-6950-1c122a"
Oct 31 04:06:16.349243 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.349205    4926 generic.go:356] "Generic (PLEG): container finished" podID="a9bd409b-1b3f-4238-a2b6-4d9a5925d728" containerID="a874f6a00f9473307295e83c7f0567e3f7305a631b27d2cb00ec4f4ea856f28a" exitCode=137
Oct 31 04:06:16.349416 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.349282    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-7a7bf7"
Oct 31 04:06:16.349416 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.349308    4926 scope.go:122] "RemoveContainer" containerID="a874f6a00f9473307295e83c7f0567e3f7305a631b27d2cb00ec4f4ea856f28a"
Oct 31 04:06:16.349640 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.349615    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6950-387ea4" podUID="7cfff2c4-773e-4523-b442-3b388f1f6894" containerName="test-curl-6950-387ea4" containerID="cri-o://6b214311329a7712c427d8b8ab315fe2c3331efc72df4c308c41855cfc41a429" gracePeriod=2
Oct 31 04:06:16.358879 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.358861    4926 scope.go:122] "RemoveContainer" containerID="a874f6a00f9473307295e83c7f0567e3f7305a631b27d2cb00ec4f4ea856f28a"
Oct 31 04:06:16.359132 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:16.359113    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a874f6a00f9473307295e83c7f0567e3f7305a631b27d2cb00ec4f4ea856f28a\": container with ID starting with a874f6a00f9473307295e83c7f0567e3f7305a631b27d2cb00ec4f4ea856f28a not found: ID does not exist" containerID="a874f6a00f9473307295e83c7f0567e3f7305a631b27d2cb00ec4f4ea856f28a"
Oct 31 04:06:16.359179 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.359142    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a874f6a00f9473307295e83c7f0567e3f7305a631b27d2cb00ec4f4ea856f28a"} err="failed to get container status \"a874f6a00f9473307295e83c7f0567e3f7305a631b27d2cb00ec4f4ea856f28a\": rpc error: code = NotFound desc = could not find container \"a874f6a00f9473307295e83c7f0567e3f7305a631b27d2cb00ec4f4ea856f28a\": container with ID starting with a874f6a00f9473307295e83c7f0567e3f7305a631b27d2cb00ec4f4ea856f28a not found: ID does not exist"
Oct 31 04:06:16.424493 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.424464    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpdk8\" (UniqueName: \"kubernetes.io/projected/81d0bdd5-01cb-4617-bca8-1928c08ff9fc-kube-api-access-cpdk8\") pod \"test-curl-6950-1c122a\" (UID: \"81d0bdd5-01cb-4617-bca8-1928c08ff9fc\") " pod="opendatahub/test-curl-6950-1c122a"
Oct 31 04:06:16.428518 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:16.428489    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-7a7bf7\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="a9bd409b-1b3f-4238-a2b6-4d9a5925d728" pod="opendatahub/test-curl-6950-7a7bf7"
Oct 31 04:06:16.481055 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:16.481015    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-7a7bf7\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="a9bd409b-1b3f-4238-a2b6-4d9a5925d728" pod="opendatahub/test-curl-6950-7a7bf7"
Oct 31 04:06:16.487467 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.487441    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpdk8\" (UniqueName: \"kubernetes.io/projected/81d0bdd5-01cb-4617-bca8-1928c08ff9fc-kube-api-access-cpdk8\") pod \"test-curl-6950-1c122a\" (UID: \"81d0bdd5-01cb-4617-bca8-1928c08ff9fc\") " pod="opendatahub/test-curl-6950-1c122a"
Oct 31 04:06:16.533633 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:16.533594    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-387ea4\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="7cfff2c4-773e-4523-b442-3b388f1f6894" pod="opendatahub/test-curl-6950-387ea4"
Oct 31 04:06:16.573184 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.573152    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-1c122a"
Oct 31 04:06:16.958672 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:06:16.958643    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81d0bdd5_01cb_4617_bca8_1928c08ff9fc.slice/crio-b24de8e4e700635eb0e80cb358e5599fcfa4580ba42ad40ccd0c2e284c3df866 WatchSource:0}: Error finding container b24de8e4e700635eb0e80cb358e5599fcfa4580ba42ad40ccd0c2e284c3df866: Status 404 returned error can't find the container with id b24de8e4e700635eb0e80cb358e5599fcfa4580ba42ad40ccd0c2e284c3df866
Oct 31 04:06:16.961482 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:16.961458    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6950-1c122a"]
Oct 31 04:06:17.358030 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:17.357936    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6950-1c122a" event={"ID":"81d0bdd5-01cb-4617-bca8-1928c08ff9fc","Type":"ContainerStarted","Data":"574d85c26cf55568deea9005646e1a6fd439be1a88eee75e5435af6524029f6e"}
Oct 31 04:06:17.358030 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:17.357975    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6950-1c122a" event={"ID":"81d0bdd5-01cb-4617-bca8-1928c08ff9fc","Type":"ContainerStarted","Data":"b24de8e4e700635eb0e80cb358e5599fcfa4580ba42ad40ccd0c2e284c3df866"}
Oct 31 04:06:17.479922 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:17.479872    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6950-1c122a" podStartSLOduration=1.47985879 podStartE2EDuration="1.47985879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:06:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:06:17.477996639 +0000 UTC m=+2274.063536901" watchObservedRunningTime="2026-10-31 04:06:17.47985879 +0000 UTC m=+2274.065399051"
Oct 31 04:06:17.531335 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:17.531295    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-7a7bf7\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="a9bd409b-1b3f-4238-a2b6-4d9a5925d728" pod="opendatahub/test-curl-6950-7a7bf7"
Oct 31 04:06:17.539041 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:17.539007    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9bd409b-1b3f-4238-a2b6-4d9a5925d728" path="/var/lib/kubelet/pods/a9bd409b-1b3f-4238-a2b6-4d9a5925d728/volumes"
Oct 31 04:06:17.584374 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:17.584337    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-387ea4\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="7cfff2c4-773e-4523-b442-3b388f1f6894" pod="opendatahub/test-curl-6950-387ea4"
Oct 31 04:06:18.583706 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:18.583683    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-387ea4"
Oct 31 04:06:18.652732 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:18.652662    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7cfff2c4-773e-4523-b442-3b388f1f6894-kube-api-access-rhl5k\" (UniqueName: \"kubernetes.io/projected/7cfff2c4-773e-4523-b442-3b388f1f6894-kube-api-access-rhl5k\") pod \"7cfff2c4-773e-4523-b442-3b388f1f6894\" (UID: \"7cfff2c4-773e-4523-b442-3b388f1f6894\") "
Oct 31 04:06:18.655047 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:18.655026    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cfff2c4-773e-4523-b442-3b388f1f6894-kube-api-access-rhl5k" pod "7cfff2c4-773e-4523-b442-3b388f1f6894" (UID: "7cfff2c4-773e-4523-b442-3b388f1f6894"). InnerVolumeSpecName "kube-api-access-rhl5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:18.753934 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:18.753899    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhl5k\" (UniqueName: \"kubernetes.io/projected/7cfff2c4-773e-4523-b442-3b388f1f6894-kube-api-access-rhl5k\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:18.944152 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:18.944067    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6950-1c122a"]
Oct 31 04:06:18.947297 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:18.947250    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6950-1c122a"]
Oct 31 04:06:19.371928 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:19.371888    4926 generic.go:356] "Generic (PLEG): container finished" podID="7cfff2c4-773e-4523-b442-3b388f1f6894" containerID="6b214311329a7712c427d8b8ab315fe2c3331efc72df4c308c41855cfc41a429" exitCode=137
Oct 31 04:06:19.372104 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:19.371940    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-387ea4"
Oct 31 04:06:19.372104 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:19.371984    4926 scope.go:122] "RemoveContainer" containerID="6b214311329a7712c427d8b8ab315fe2c3331efc72df4c308c41855cfc41a429"
Oct 31 04:06:19.372235 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:19.372152    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6950-1c122a" podUID="81d0bdd5-01cb-4617-bca8-1928c08ff9fc" containerName="test-curl-6950-1c122a" containerID="cri-o://574d85c26cf55568deea9005646e1a6fd439be1a88eee75e5435af6524029f6e" gracePeriod=2
Oct 31 04:06:19.381768 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:19.381736    4926 scope.go:122] "RemoveContainer" containerID="6b214311329a7712c427d8b8ab315fe2c3331efc72df4c308c41855cfc41a429"
Oct 31 04:06:19.382069 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:19.382048    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b214311329a7712c427d8b8ab315fe2c3331efc72df4c308c41855cfc41a429\": container with ID starting with 6b214311329a7712c427d8b8ab315fe2c3331efc72df4c308c41855cfc41a429 not found: ID does not exist" containerID="6b214311329a7712c427d8b8ab315fe2c3331efc72df4c308c41855cfc41a429"
Oct 31 04:06:19.382144 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:19.382078    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b214311329a7712c427d8b8ab315fe2c3331efc72df4c308c41855cfc41a429"} err="failed to get container status \"6b214311329a7712c427d8b8ab315fe2c3331efc72df4c308c41855cfc41a429\": rpc error: code = NotFound desc = could not find container \"6b214311329a7712c427d8b8ab315fe2c3331efc72df4c308c41855cfc41a429\": container with ID starting with 6b214311329a7712c427d8b8ab315fe2c3331efc72df4c308c41855cfc41a429 not found: ID does not exist"
Oct 31 04:06:19.431290 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:19.428513    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-1c122a\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="81d0bdd5-01cb-4617-bca8-1928c08ff9fc" pod="opendatahub/test-curl-6950-1c122a"
Oct 31 04:06:19.483907 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:19.483846    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-1c122a\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="81d0bdd5-01cb-4617-bca8-1928c08ff9fc" pod="opendatahub/test-curl-6950-1c122a"
Oct 31 04:06:19.536724 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:19.536682    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-387ea4\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="7cfff2c4-773e-4523-b442-3b388f1f6894" pod="opendatahub/test-curl-6950-387ea4"
Oct 31 04:06:19.539540 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:19.539512    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7cfff2c4-773e-4523-b442-3b388f1f6894" path="/var/lib/kubelet/pods/7cfff2c4-773e-4523-b442-3b388f1f6894/volumes"
Oct 31 04:06:21.601476 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:21.601451    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-1c122a"
Oct 31 04:06:21.693829 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:21.693625    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81d0bdd5-01cb-4617-bca8-1928c08ff9fc-kube-api-access-cpdk8\" (UniqueName: \"kubernetes.io/projected/81d0bdd5-01cb-4617-bca8-1928c08ff9fc-kube-api-access-cpdk8\") pod \"81d0bdd5-01cb-4617-bca8-1928c08ff9fc\" (UID: \"81d0bdd5-01cb-4617-bca8-1928c08ff9fc\") "
Oct 31 04:06:21.696995 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:21.696949    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81d0bdd5-01cb-4617-bca8-1928c08ff9fc-kube-api-access-cpdk8" pod "81d0bdd5-01cb-4617-bca8-1928c08ff9fc" (UID: "81d0bdd5-01cb-4617-bca8-1928c08ff9fc"). InnerVolumeSpecName "kube-api-access-cpdk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:21.795341 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:21.795301    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpdk8\" (UniqueName: \"kubernetes.io/projected/81d0bdd5-01cb-4617-bca8-1928c08ff9fc-kube-api-access-cpdk8\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:22.390710 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:22.390671    4926 generic.go:356] "Generic (PLEG): container finished" podID="81d0bdd5-01cb-4617-bca8-1928c08ff9fc" containerID="574d85c26cf55568deea9005646e1a6fd439be1a88eee75e5435af6524029f6e" exitCode=137
Oct 31 04:06:22.390889 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:22.390712    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-1c122a"
Oct 31 04:06:22.390889 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:22.390723    4926 scope.go:122] "RemoveContainer" containerID="574d85c26cf55568deea9005646e1a6fd439be1a88eee75e5435af6524029f6e"
Oct 31 04:06:22.400317 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:22.400286    4926 scope.go:122] "RemoveContainer" containerID="574d85c26cf55568deea9005646e1a6fd439be1a88eee75e5435af6524029f6e"
Oct 31 04:06:22.400545 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:22.400529    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"574d85c26cf55568deea9005646e1a6fd439be1a88eee75e5435af6524029f6e\": container with ID starting with 574d85c26cf55568deea9005646e1a6fd439be1a88eee75e5435af6524029f6e not found: ID does not exist" containerID="574d85c26cf55568deea9005646e1a6fd439be1a88eee75e5435af6524029f6e"
Oct 31 04:06:22.400602 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:22.400552    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"574d85c26cf55568deea9005646e1a6fd439be1a88eee75e5435af6524029f6e"} err="failed to get container status \"574d85c26cf55568deea9005646e1a6fd439be1a88eee75e5435af6524029f6e\": rpc error: code = NotFound desc = could not find container \"574d85c26cf55568deea9005646e1a6fd439be1a88eee75e5435af6524029f6e\": container with ID starting with 574d85c26cf55568deea9005646e1a6fd439be1a88eee75e5435af6524029f6e not found: ID does not exist"
Oct 31 04:06:22.451477 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:22.451440    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-1c122a\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="81d0bdd5-01cb-4617-bca8-1928c08ff9fc" pod="opendatahub/test-curl-6950-1c122a"
Oct 31 04:06:23.539201 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:23.539165    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81d0bdd5-01cb-4617-bca8-1928c08ff9fc" path="/var/lib/kubelet/pods/81d0bdd5-01cb-4617-bca8-1928c08ff9fc/volumes"
Oct 31 04:06:23.587584 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:23.587550    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-1c122a\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="81d0bdd5-01cb-4617-bca8-1928c08ff9fc" pod="opendatahub/test-curl-6950-1c122a"
Oct 31 04:06:23.988187 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:23.988155    4926 scope.go:122] "RemoveContainer" containerID="76f8e9a679714ae42f643032203cf655542e664add8fb0b0bb434be47c9cc263"
Oct 31 04:06:24.817681 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:24.817637    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"]
Oct 31 04:06:29.105758 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:29.105726    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6950-41698b"]
Oct 31 04:06:29.106199 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:29.106186    4926 cpu_manager.go:414] "containerMap: removing container" podUID="81d0bdd5-01cb-4617-bca8-1928c08ff9fc" containerName="test-curl-6950-1c122a"
Oct 31 04:06:29.106246 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:29.106200    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81d0bdd5-01cb-4617-bca8-1928c08ff9fc" containerName="test-curl-6950-1c122a"
Oct 31 04:06:29.106316 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:29.106308    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="81d0bdd5-01cb-4617-bca8-1928c08ff9fc" containerName="test-curl-6950-1c122a"
Oct 31 04:06:29.111076 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:29.111059    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-41698b"
Oct 31 04:06:29.162048 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:29.162014    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6950-41698b"]
Oct 31 04:06:29.244291 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:29.242503    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-2x2q5\""
Oct 31 04:06:29.285110 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:29.285074    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tqc5\" (UniqueName: \"kubernetes.io/projected/6815167d-3148-404f-b12f-c338910533b4-kube-api-access-5tqc5\") pod \"test-curl-6950-41698b\" (UID: \"6815167d-3148-404f-b12f-c338910533b4\") " pod="opendatahub/test-curl-6950-41698b"
Oct 31 04:06:29.386844 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:29.386749    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tqc5\" (UniqueName: \"kubernetes.io/projected/6815167d-3148-404f-b12f-c338910533b4-kube-api-access-5tqc5\") pod \"test-curl-6950-41698b\" (UID: \"6815167d-3148-404f-b12f-c338910533b4\") " pod="opendatahub/test-curl-6950-41698b"
Oct 31 04:06:29.450107 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:29.450077    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tqc5\" (UniqueName: \"kubernetes.io/projected/6815167d-3148-404f-b12f-c338910533b4-kube-api-access-5tqc5\") pod \"test-curl-6950-41698b\" (UID: \"6815167d-3148-404f-b12f-c338910533b4\") " pod="opendatahub/test-curl-6950-41698b"
Oct 31 04:06:29.723442 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:29.723411    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-41698b"
Oct 31 04:06:30.105708 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:06:30.105658    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6815167d_3148_404f_b12f_c338910533b4.slice/crio-a0ecc64452f2bcaaa695590277ba22f791ab51f6f55963d26b3bab86282ade74 WatchSource:0}: Error finding container a0ecc64452f2bcaaa695590277ba22f791ab51f6f55963d26b3bab86282ade74: Status 404 returned error can't find the container with id a0ecc64452f2bcaaa695590277ba22f791ab51f6f55963d26b3bab86282ade74
Oct 31 04:06:30.108622 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:30.108593    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6950-41698b"]
Oct 31 04:06:30.438650 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:30.438559    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6950-41698b" event={"ID":"6815167d-3148-404f-b12f-c338910533b4","Type":"ContainerStarted","Data":"62ca9fa1c3a25fb4809d81c326411fd08d03ebf0bf0e02c7cb529beda3ebacec"}
Oct 31 04:06:30.438650 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:30.438610    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6950-41698b" event={"ID":"6815167d-3148-404f-b12f-c338910533b4","Type":"ContainerStarted","Data":"a0ecc64452f2bcaaa695590277ba22f791ab51f6f55963d26b3bab86282ade74"}
Oct 31 04:06:30.557289 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:30.557229    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6950-41698b" podStartSLOduration=1.5572157340000001 podStartE2EDuration="1.557215734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:06:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:06:30.555158269 +0000 UTC m=+2287.140698532" watchObservedRunningTime="2026-10-31 04:06:30.557215734 +0000 UTC m=+2287.142755996"
Oct 31 04:06:35.313509 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:35.313466    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:06:35.410335 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:35.410298    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"]
Oct 31 04:06:35.463183 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:35.463140    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"]
Oct 31 04:06:35.467053 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:35.467018    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"]
Oct 31 04:06:40.430112 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.430037    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" podUID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" containerName="main" containerID="cri-o://47f4c01c7633c2700a9689e3df6b6f6c639ed7429a310dfd3fc4b3933b92b93e" gracePeriod=45
Oct 31 04:06:40.525106 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.524804    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" podUID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": dial tcp 10.132.0.45:8000: connect: connection refused"
Oct 31 04:06:40.696938 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.696878    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 04:06:40.733663 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.733638    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-kube-api-access-7wvtz\" (UniqueName: \"kubernetes.io/projected/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-kube-api-access-7wvtz\") pod \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") "
Oct 31 04:06:40.733816 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.733742    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-home\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-home\") pod \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") "
Oct 31 04:06:40.733855 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.733821    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-tmp-dir\") pod \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") "
Oct 31 04:06:40.733855 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.733847    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-tls-certs\" (UniqueName: \"kubernetes.io/secret/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-tls-certs\") pod \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") "
Oct 31 04:06:40.733919 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.733880    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-dshm\") pod \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") "
Oct 31 04:06:40.733968 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.733950    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-model-cache\") pod \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\" (UID: \"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f\") "
Oct 31 04:06:40.734022 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.733991    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-home" pod "fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" (UID: "fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:40.734148 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.734128    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-tmp-dir" pod "fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" (UID: "fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:40.734312 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.734259    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-model-cache" pod "fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" (UID: "fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:40.734503 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.734487    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:40.734566 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.734508    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:40.734566 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.734523    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:40.736140 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.736118    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-kube-api-access-7wvtz" pod "fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" (UID: "fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f"). InnerVolumeSpecName "kube-api-access-7wvtz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:40.736216 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.736123    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-dshm" pod "fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" (UID: "fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:40.736260 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.736209    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-tls-certs" pod "fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" (UID: "fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:06:40.825668 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.825611    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6950-41698b"]
Oct 31 04:06:40.825888 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.825836    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6950-41698b" podUID="6815167d-3148-404f-b12f-c338910533b4" containerName="test-curl-6950-41698b" containerID="cri-o://62ca9fa1c3a25fb4809d81c326411fd08d03ebf0bf0e02c7cb529beda3ebacec" gracePeriod=2
Oct 31 04:06:40.830888 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.830859    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6950-41698b"]
Oct 31 04:06:40.836068 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.836039    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wvtz\" (UniqueName: \"kubernetes.io/projected/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-kube-api-access-7wvtz\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:40.836174 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.836075    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:40.836174 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:40.836092    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:40.878709 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:40.878671    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-41698b\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="6815167d-3148-404f-b12f-c338910533b4" pod="opendatahub/test-curl-6950-41698b"
Oct 31 04:06:41.505618 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:41.505582    4926 generic.go:356] "Generic (PLEG): container finished" podID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" containerID="47f4c01c7633c2700a9689e3df6b6f6c639ed7429a310dfd3fc4b3933b92b93e" exitCode=0
Oct 31 04:06:41.506092 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:41.505642    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"
Oct 31 04:06:41.506092 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:41.505673    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" event={"ID":"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f","Type":"ContainerDied","Data":"47f4c01c7633c2700a9689e3df6b6f6c639ed7429a310dfd3fc4b3933b92b93e"}
Oct 31 04:06:41.506092 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:41.505730    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w" event={"ID":"fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f","Type":"ContainerDied","Data":"cec4fac8a97f34987c077df63de100a987582680aa1790c1874ffd71302d7a28"}
Oct 31 04:06:41.506092 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:41.505757    4926 scope.go:122] "RemoveContainer" containerID="47f4c01c7633c2700a9689e3df6b6f6c639ed7429a310dfd3fc4b3933b92b93e"
Oct 31 04:06:41.515592 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:41.515572    4926 scope.go:122] "RemoveContainer" containerID="47f4c01c7633c2700a9689e3df6b6f6c639ed7429a310dfd3fc4b3933b92b93e"
Oct 31 04:06:41.515859 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:41.515836    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47f4c01c7633c2700a9689e3df6b6f6c639ed7429a310dfd3fc4b3933b92b93e\": container with ID starting with 47f4c01c7633c2700a9689e3df6b6f6c639ed7429a310dfd3fc4b3933b92b93e not found: ID does not exist" containerID="47f4c01c7633c2700a9689e3df6b6f6c639ed7429a310dfd3fc4b3933b92b93e"
Oct 31 04:06:41.515907 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:41.515873    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47f4c01c7633c2700a9689e3df6b6f6c639ed7429a310dfd3fc4b3933b92b93e"} err="failed to get container status \"47f4c01c7633c2700a9689e3df6b6f6c639ed7429a310dfd3fc4b3933b92b93e\": rpc error: code = NotFound desc = could not find container \"47f4c01c7633c2700a9689e3df6b6f6c639ed7429a310dfd3fc4b3933b92b93e\": container with ID starting with 47f4c01c7633c2700a9689e3df6b6f6c639ed7429a310dfd3fc4b3933b92b93e not found: ID does not exist"
Oct 31 04:06:41.558543 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:41.558491    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-41698b\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="6815167d-3148-404f-b12f-c338910533b4" pod="opendatahub/test-curl-6950-41698b"
Oct 31 04:06:41.727524 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:41.727488    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"]
Oct 31 04:06:41.778628 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:41.778546    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-d7b7bc/simulator-w1-d7b7bc-kserve-8577d654f5-2qb8w"]
Oct 31 04:06:43.107759 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:43.107734    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-41698b"
Oct 31 04:06:43.164160 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:43.164124    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6815167d-3148-404f-b12f-c338910533b4-kube-api-access-5tqc5\" (UniqueName: \"kubernetes.io/projected/6815167d-3148-404f-b12f-c338910533b4-kube-api-access-5tqc5\") pod \"6815167d-3148-404f-b12f-c338910533b4\" (UID: \"6815167d-3148-404f-b12f-c338910533b4\") "
Oct 31 04:06:43.166673 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:43.166635    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6815167d-3148-404f-b12f-c338910533b4-kube-api-access-5tqc5" pod "6815167d-3148-404f-b12f-c338910533b4" (UID: "6815167d-3148-404f-b12f-c338910533b4"). InnerVolumeSpecName "kube-api-access-5tqc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:43.265811 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:43.265779    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tqc5\" (UniqueName: \"kubernetes.io/projected/6815167d-3148-404f-b12f-c338910533b4-kube-api-access-5tqc5\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:43.519801 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:43.519765    4926 generic.go:356] "Generic (PLEG): container finished" podID="6815167d-3148-404f-b12f-c338910533b4" containerID="62ca9fa1c3a25fb4809d81c326411fd08d03ebf0bf0e02c7cb529beda3ebacec" exitCode=137
Oct 31 04:06:43.519982 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:43.519809    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6950-41698b"
Oct 31 04:06:43.519982 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:43.519855    4926 scope.go:122] "RemoveContainer" containerID="62ca9fa1c3a25fb4809d81c326411fd08d03ebf0bf0e02c7cb529beda3ebacec"
Oct 31 04:06:43.529380 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:43.529360    4926 scope.go:122] "RemoveContainer" containerID="62ca9fa1c3a25fb4809d81c326411fd08d03ebf0bf0e02c7cb529beda3ebacec"
Oct 31 04:06:43.529658 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:43.529639    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62ca9fa1c3a25fb4809d81c326411fd08d03ebf0bf0e02c7cb529beda3ebacec\": container with ID starting with 62ca9fa1c3a25fb4809d81c326411fd08d03ebf0bf0e02c7cb529beda3ebacec not found: ID does not exist" containerID="62ca9fa1c3a25fb4809d81c326411fd08d03ebf0bf0e02c7cb529beda3ebacec"
Oct 31 04:06:43.529731 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:43.529669    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62ca9fa1c3a25fb4809d81c326411fd08d03ebf0bf0e02c7cb529beda3ebacec"} err="failed to get container status \"62ca9fa1c3a25fb4809d81c326411fd08d03ebf0bf0e02c7cb529beda3ebacec\": rpc error: code = NotFound desc = could not find container \"62ca9fa1c3a25fb4809d81c326411fd08d03ebf0bf0e02c7cb529beda3ebacec\": container with ID starting with 62ca9fa1c3a25fb4809d81c326411fd08d03ebf0bf0e02c7cb529beda3ebacec not found: ID does not exist"
Oct 31 04:06:43.538315 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:43.538294    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6815167d-3148-404f-b12f-c338910533b4" path="/var/lib/kubelet/pods/6815167d-3148-404f-b12f-c338910533b4/volumes"
Oct 31 04:06:43.538752 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:43.538738    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" path="/var/lib/kubelet/pods/fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f/volumes"
Oct 31 04:06:43.580917 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:43.580893    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6950-41698b\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="6815167d-3148-404f-b12f-c338910533b4" pod="opendatahub/test-curl-6950-41698b"
Oct 31 04:06:50.431409 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.431352    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59" podUID="4b319682-afe9-4edd-b9c0-46d7cb7a989c" containerName="main" containerID="cri-o://637be0d48342e518ce989be5370f34f1061ce0b2b0bdddb3252baa127b5c5022" gracePeriod=45
Oct 31 04:06:50.488981 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.488920    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerName="main" containerID="cri-o://60328cd17c6eb4f894f4458ea1c38118fcfaf7e67d0f928c6103dce9a05fbfe6" gracePeriod=45
Oct 31 04:06:50.491522 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.491486    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" podUID="564888f8-35bb-4028-8040-110c959c9b19" containerName="main" containerID="cri-o://f0692352bbbc5d76ee0616daec4d58c37059fc518e73ebf590f6203ddef530e3" gracePeriod=45
Oct 31 04:06:50.557770 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.557739    4926 generic.go:356] "Generic (PLEG): container finished" podID="4b319682-afe9-4edd-b9c0-46d7cb7a989c" containerID="637be0d48342e518ce989be5370f34f1061ce0b2b0bdddb3252baa127b5c5022" exitCode=0
Oct 31 04:06:50.557928 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.557803    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59" event={"ID":"4b319682-afe9-4edd-b9c0-46d7cb7a989c","Type":"ContainerDied","Data":"637be0d48342e518ce989be5370f34f1061ce0b2b0bdddb3252baa127b5c5022"}
Oct 31 04:06:50.785605 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.785580    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 04:06:50.787814 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.787791    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 04:06:50.798537 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.798513    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 04:06:50.865674 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.865636    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3f4959e7-6fb3-4687-8d65-664ef92519f5-tls-certs\" (UniqueName: \"kubernetes.io/secret/3f4959e7-6fb3-4687-8d65-664ef92519f5-tls-certs\") pod \"3f4959e7-6fb3-4687-8d65-664ef92519f5\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") "
Oct 31 04:06:50.865674 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.865675    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-tmp-dir\") pod \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") "
Oct 31 04:06:50.865909 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.865708    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b319682-afe9-4edd-b9c0-46d7cb7a989c-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b319682-afe9-4edd-b9c0-46d7cb7a989c-tls-certs\") pod \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") "
Oct 31 04:06:50.865909 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.865758    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/564888f8-35bb-4028-8040-110c959c9b19-kube-api-access-svcx6\" (UniqueName: \"kubernetes.io/projected/564888f8-35bb-4028-8040-110c959c9b19-kube-api-access-svcx6\") pod \"564888f8-35bb-4028-8040-110c959c9b19\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") "
Oct 31 04:06:50.865909 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.865804    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-model-cache\") pod \"564888f8-35bb-4028-8040-110c959c9b19\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") "
Oct 31 04:06:50.865909 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.865840    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-home\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-home\") pod \"564888f8-35bb-4028-8040-110c959c9b19\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") "
Oct 31 04:06:50.865909 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.865902    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-home\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-home\") pod \"3f4959e7-6fb3-4687-8d65-664ef92519f5\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") "
Oct 31 04:06:50.866189 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.865971    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-dshm\") pod \"3f4959e7-6fb3-4687-8d65-664ef92519f5\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") "
Oct 31 04:06:50.866189 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.865999    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-dshm\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-dshm\") pod \"564888f8-35bb-4028-8040-110c959c9b19\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") "
Oct 31 04:06:50.866189 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.866004    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-tmp-dir" pod "4b319682-afe9-4edd-b9c0-46d7cb7a989c" (UID: "4b319682-afe9-4edd-b9c0-46d7cb7a989c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:50.866189 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.866042    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-dshm\") pod \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") "
Oct 31 04:06:50.866189 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.866114    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/564888f8-35bb-4028-8040-110c959c9b19-tls-certs\" (UniqueName: \"kubernetes.io/secret/564888f8-35bb-4028-8040-110c959c9b19-tls-certs\") pod \"564888f8-35bb-4028-8040-110c959c9b19\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") "
Oct 31 04:06:50.866189 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.866137    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-home\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-home\") pod \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") "
Oct 31 04:06:50.866527 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.866216    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-tmp-dir\") pod \"564888f8-35bb-4028-8040-110c959c9b19\" (UID: \"564888f8-35bb-4028-8040-110c959c9b19\") "
Oct 31 04:06:50.866527 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.866244    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-home" pod "3f4959e7-6fb3-4687-8d65-664ef92519f5" (UID: "3f4959e7-6fb3-4687-8d65-664ef92519f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:50.866527 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.866247    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b319682-afe9-4edd-b9c0-46d7cb7a989c-kube-api-access-qvk8r\" (UniqueName: \"kubernetes.io/projected/4b319682-afe9-4edd-b9c0-46d7cb7a989c-kube-api-access-qvk8r\") pod \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") "
Oct 31 04:06:50.866527 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.866319    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-model-cache\") pod \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\" (UID: \"4b319682-afe9-4edd-b9c0-46d7cb7a989c\") "
Oct 31 04:06:50.866527 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.866387    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-tmp-dir\") pod \"3f4959e7-6fb3-4687-8d65-664ef92519f5\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") "
Oct 31 04:06:50.866527 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.866414    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-model-cache" pod "564888f8-35bb-4028-8040-110c959c9b19" (UID: "564888f8-35bb-4028-8040-110c959c9b19"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:50.866527 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.866419    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f4959e7-6fb3-4687-8d65-664ef92519f5-kube-api-access-xpcxg\" (UniqueName: \"kubernetes.io/projected/3f4959e7-6fb3-4687-8d65-664ef92519f5-kube-api-access-xpcxg\") pod \"3f4959e7-6fb3-4687-8d65-664ef92519f5\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") "
Oct 31 04:06:50.866898 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.866758    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-model-cache\") pod \"3f4959e7-6fb3-4687-8d65-664ef92519f5\" (UID: \"3f4959e7-6fb3-4687-8d65-664ef92519f5\") "
Oct 31 04:06:50.866898 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.866866    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-home" pod "564888f8-35bb-4028-8040-110c959c9b19" (UID: "564888f8-35bb-4028-8040-110c959c9b19"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:50.867741 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.867556    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.867741 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.867581    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.867741 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.867596    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.867741 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.867612    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.867741 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.867700    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-home" pod "4b319682-afe9-4edd-b9c0-46d7cb7a989c" (UID: "4b319682-afe9-4edd-b9c0-46d7cb7a989c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:50.868105 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.867819    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-model-cache" pod "3f4959e7-6fb3-4687-8d65-664ef92519f5" (UID: "3f4959e7-6fb3-4687-8d65-664ef92519f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:50.868105 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.867839    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-tmp-dir" pod "564888f8-35bb-4028-8040-110c959c9b19" (UID: "564888f8-35bb-4028-8040-110c959c9b19"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:50.868105 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.867916    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-tmp-dir" pod "3f4959e7-6fb3-4687-8d65-664ef92519f5" (UID: "3f4959e7-6fb3-4687-8d65-664ef92519f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:50.868105 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.867973    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-model-cache" pod "4b319682-afe9-4edd-b9c0-46d7cb7a989c" (UID: "4b319682-afe9-4edd-b9c0-46d7cb7a989c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:50.869731 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.869673    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/564888f8-35bb-4028-8040-110c959c9b19-tls-certs" pod "564888f8-35bb-4028-8040-110c959c9b19" (UID: "564888f8-35bb-4028-8040-110c959c9b19"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:06:50.870043 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.870020    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-dshm" pod "564888f8-35bb-4028-8040-110c959c9b19" (UID: "564888f8-35bb-4028-8040-110c959c9b19"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:50.870122 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.870048    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b319682-afe9-4edd-b9c0-46d7cb7a989c-tls-certs" pod "4b319682-afe9-4edd-b9c0-46d7cb7a989c" (UID: "4b319682-afe9-4edd-b9c0-46d7cb7a989c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:06:50.870122 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.870049    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f4959e7-6fb3-4687-8d65-664ef92519f5-kube-api-access-xpcxg" pod "3f4959e7-6fb3-4687-8d65-664ef92519f5" (UID: "3f4959e7-6fb3-4687-8d65-664ef92519f5"). InnerVolumeSpecName "kube-api-access-xpcxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:50.870193 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.870175    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b319682-afe9-4edd-b9c0-46d7cb7a989c-kube-api-access-qvk8r" pod "4b319682-afe9-4edd-b9c0-46d7cb7a989c" (UID: "4b319682-afe9-4edd-b9c0-46d7cb7a989c"). InnerVolumeSpecName "kube-api-access-qvk8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:50.870193 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.870186    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/564888f8-35bb-4028-8040-110c959c9b19-kube-api-access-svcx6" pod "564888f8-35bb-4028-8040-110c959c9b19" (UID: "564888f8-35bb-4028-8040-110c959c9b19"). InnerVolumeSpecName "kube-api-access-svcx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:06:50.870373 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.870357    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-dshm" pod "3f4959e7-6fb3-4687-8d65-664ef92519f5" (UID: "3f4959e7-6fb3-4687-8d65-664ef92519f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:50.870462 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.870445    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f4959e7-6fb3-4687-8d65-664ef92519f5-tls-certs" pod "3f4959e7-6fb3-4687-8d65-664ef92519f5" (UID: "3f4959e7-6fb3-4687-8d65-664ef92519f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:06:50.871759 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.871743    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-dshm" pod "4b319682-afe9-4edd-b9c0-46d7cb7a989c" (UID: "4b319682-afe9-4edd-b9c0-46d7cb7a989c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:06:50.968407 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.968379    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.968407 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.968410    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvk8r\" (UniqueName: \"kubernetes.io/projected/4b319682-afe9-4edd-b9c0-46d7cb7a989c-kube-api-access-qvk8r\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.968621 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.968424    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.968621 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.968436    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.968621 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.968445    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpcxg\" (UniqueName: \"kubernetes.io/projected/3f4959e7-6fb3-4687-8d65-664ef92519f5-kube-api-access-xpcxg\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.968621 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.968453    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.968621 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.968462    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f4959e7-6fb3-4687-8d65-664ef92519f5-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.968621 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.968470    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b319682-afe9-4edd-b9c0-46d7cb7a989c-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.968621 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.968477    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svcx6\" (UniqueName: \"kubernetes.io/projected/564888f8-35bb-4028-8040-110c959c9b19-kube-api-access-svcx6\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.968621 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.968486    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3f4959e7-6fb3-4687-8d65-664ef92519f5-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.968621 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.968494    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/564888f8-35bb-4028-8040-110c959c9b19-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.968621 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.968501    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.968621 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.968509    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/564888f8-35bb-4028-8040-110c959c9b19-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:50.968621 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:50.968515    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b319682-afe9-4edd-b9c0-46d7cb7a989c-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:06:51.564083 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.564052    4926 generic.go:356] "Generic (PLEG): container finished" podID="564888f8-35bb-4028-8040-110c959c9b19" containerID="f0692352bbbc5d76ee0616daec4d58c37059fc518e73ebf590f6203ddef530e3" exitCode=0
Oct 31 04:06:51.564455 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.564094    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"
Oct 31 04:06:51.564455 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.564136    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" event={"ID":"564888f8-35bb-4028-8040-110c959c9b19","Type":"ContainerDied","Data":"f0692352bbbc5d76ee0616daec4d58c37059fc518e73ebf590f6203ddef530e3"}
Oct 31 04:06:51.564455 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.564172    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw" event={"ID":"564888f8-35bb-4028-8040-110c959c9b19","Type":"ContainerDied","Data":"21554e32e73743d263944a51e3af54a7501ab39950a0db216bb9a4c952cf4424"}
Oct 31 04:06:51.564455 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.564187    4926 scope.go:122] "RemoveContainer" containerID="f0692352bbbc5d76ee0616daec4d58c37059fc518e73ebf590f6203ddef530e3"
Oct 31 04:06:51.565673 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.565649    4926 generic.go:356] "Generic (PLEG): container finished" podID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerID="60328cd17c6eb4f894f4458ea1c38118fcfaf7e67d0f928c6103dce9a05fbfe6" exitCode=0
Oct 31 04:06:51.565758 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.565706    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"
Oct 31 04:06:51.565805 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.565761    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" event={"ID":"3f4959e7-6fb3-4687-8d65-664ef92519f5","Type":"ContainerDied","Data":"60328cd17c6eb4f894f4458ea1c38118fcfaf7e67d0f928c6103dce9a05fbfe6"}
Oct 31 04:06:51.565805 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.565794    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr" event={"ID":"3f4959e7-6fb3-4687-8d65-664ef92519f5","Type":"ContainerDied","Data":"071d0eb419b25a65f054e159f3340de7eb6c63011159a1207d6e0eaa31e25296"}
Oct 31 04:06:51.567457 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.567425    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59" event={"ID":"4b319682-afe9-4edd-b9c0-46d7cb7a989c","Type":"ContainerDied","Data":"698f230b627d5e11895ce9f9111b04d60a0aa4d3a2819bb142b842b7c687ebfb"}
Oct 31 04:06:51.567457 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.567435    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"
Oct 31 04:06:51.574005 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.573987    4926 scope.go:122] "RemoveContainer" containerID="f0692352bbbc5d76ee0616daec4d58c37059fc518e73ebf590f6203ddef530e3"
Oct 31 04:06:51.574301 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:51.574258    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0692352bbbc5d76ee0616daec4d58c37059fc518e73ebf590f6203ddef530e3\": container with ID starting with f0692352bbbc5d76ee0616daec4d58c37059fc518e73ebf590f6203ddef530e3 not found: ID does not exist" containerID="f0692352bbbc5d76ee0616daec4d58c37059fc518e73ebf590f6203ddef530e3"
Oct 31 04:06:51.574339 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.574310    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0692352bbbc5d76ee0616daec4d58c37059fc518e73ebf590f6203ddef530e3"} err="failed to get container status \"f0692352bbbc5d76ee0616daec4d58c37059fc518e73ebf590f6203ddef530e3\": rpc error: code = NotFound desc = could not find container \"f0692352bbbc5d76ee0616daec4d58c37059fc518e73ebf590f6203ddef530e3\": container with ID starting with f0692352bbbc5d76ee0616daec4d58c37059fc518e73ebf590f6203ddef530e3 not found: ID does not exist"
Oct 31 04:06:51.574339 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.574328    4926 scope.go:122] "RemoveContainer" containerID="60328cd17c6eb4f894f4458ea1c38118fcfaf7e67d0f928c6103dce9a05fbfe6"
Oct 31 04:06:51.582428 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.582416    4926 scope.go:122] "RemoveContainer" containerID="60328cd17c6eb4f894f4458ea1c38118fcfaf7e67d0f928c6103dce9a05fbfe6"
Oct 31 04:06:51.582699 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:06:51.582682    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60328cd17c6eb4f894f4458ea1c38118fcfaf7e67d0f928c6103dce9a05fbfe6\": container with ID starting with 60328cd17c6eb4f894f4458ea1c38118fcfaf7e67d0f928c6103dce9a05fbfe6 not found: ID does not exist" containerID="60328cd17c6eb4f894f4458ea1c38118fcfaf7e67d0f928c6103dce9a05fbfe6"
Oct 31 04:06:51.582740 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.582704    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60328cd17c6eb4f894f4458ea1c38118fcfaf7e67d0f928c6103dce9a05fbfe6"} err="failed to get container status \"60328cd17c6eb4f894f4458ea1c38118fcfaf7e67d0f928c6103dce9a05fbfe6\": rpc error: code = NotFound desc = could not find container \"60328cd17c6eb4f894f4458ea1c38118fcfaf7e67d0f928c6103dce9a05fbfe6\": container with ID starting with 60328cd17c6eb4f894f4458ea1c38118fcfaf7e67d0f928c6103dce9a05fbfe6 not found: ID does not exist"
Oct 31 04:06:51.582740 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.582720    4926 scope.go:122] "RemoveContainer" containerID="637be0d48342e518ce989be5370f34f1061ce0b2b0bdddb3252baa127b5c5022"
Oct 31 04:06:51.737355 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.737318    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"]
Oct 31 04:06:51.739741 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.739714    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-537e34/distinct-2-w3-537e34-kserve-68448d4986-95r59"]
Oct 31 04:06:51.851871 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.851839    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"]
Oct 31 04:06:51.856167 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.856143    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-537e34/simulator-w3-537e34-kserve-9c8459cff-n2znw"]
Oct 31 04:06:51.968015 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.967982    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"]
Oct 31 04:06:51.971895 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:51.971871    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-537e34/distinct-w3-537e34-kserve-69b646b9bf-xjbwr"]
Oct 31 04:06:53.544410 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:53.544378    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" path="/var/lib/kubelet/pods/3f4959e7-6fb3-4687-8d65-664ef92519f5/volumes"
Oct 31 04:06:53.544794 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:53.544780    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b319682-afe9-4edd-b9c0-46d7cb7a989c" path="/var/lib/kubelet/pods/4b319682-afe9-4edd-b9c0-46d7cb7a989c/volumes"
Oct 31 04:06:53.545166 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:06:53.545152    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="564888f8-35bb-4028-8040-110c959c9b19" path="/var/lib/kubelet/pods/564888f8-35bb-4028-8040-110c959c9b19/volumes"
Oct 31 04:07:05.314916 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:07:05.314877    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:07:07.128902 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.128867    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"]
Oct 31 04:07:07.129598 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.129576    4926 cpu_manager.go:414] "containerMap: removing container" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerName="main"
Oct 31 04:07:07.129694 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.129600    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerName="main"
Oct 31 04:07:07.129694 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.129653    4926 cpu_manager.go:414] "containerMap: removing container" podUID="4b319682-afe9-4edd-b9c0-46d7cb7a989c" containerName="main"
Oct 31 04:07:07.129694 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.129661    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b319682-afe9-4edd-b9c0-46d7cb7a989c" containerName="main"
Oct 31 04:07:07.129694 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.129673    4926 cpu_manager.go:414] "containerMap: removing container" podUID="6815167d-3148-404f-b12f-c338910533b4" containerName="test-curl-6950-41698b"
Oct 31 04:07:07.129694 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.129682    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6815167d-3148-404f-b12f-c338910533b4" containerName="test-curl-6950-41698b"
Oct 31 04:07:07.129948 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.129696    4926 cpu_manager.go:414] "containerMap: removing container" podUID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" containerName="main"
Oct 31 04:07:07.129948 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.129703    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" containerName="main"
Oct 31 04:07:07.129948 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.129735    4926 cpu_manager.go:414] "containerMap: removing container" podUID="564888f8-35bb-4028-8040-110c959c9b19" containerName="main"
Oct 31 04:07:07.129948 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.129743    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="564888f8-35bb-4028-8040-110c959c9b19" containerName="main"
Oct 31 04:07:07.129948 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.129883    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b319682-afe9-4edd-b9c0-46d7cb7a989c" containerName="main"
Oct 31 04:07:07.129948 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.129896    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="6815167d-3148-404f-b12f-c338910533b4" containerName="test-curl-6950-41698b"
Oct 31 04:07:07.129948 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.129905    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="fde9e7f9-5a81-4c77-b5ee-7996f5a6a48f" containerName="main"
Oct 31 04:07:07.129948 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.129920    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="564888f8-35bb-4028-8040-110c959c9b19" containerName="main"
Oct 31 04:07:07.129948 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.129933    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f4959e7-6fb3-4687-8d65-664ef92519f5" containerName="main"
Oct 31 04:07:07.135105 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.135082    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.173778 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.173740    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-model-cache\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.173943 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.173853    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-dshm\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.173943 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.173887    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t556\" (UniqueName: \"kubernetes.io/projected/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-kube-api-access-4t556\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.173943 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.173920    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-tls-certs\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.174049 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.174006    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-home\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.174190 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.174175    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-tmp-dir\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.188693 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.188663    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"]
Oct 31 04:07:07.236330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.236298    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6e81aea5\"/\"kube-root-ca.crt\""
Oct 31 04:07:07.236519 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.236370    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-6e81aea5\"/\"openshift-service-ca.crt\""
Oct 31 04:07:07.240426 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.240402    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6e81aea5\"/\"e2e-lifecycle-model-6e81aea5-kserve-self-signed-certs\""
Oct 31 04:07:07.240537 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.240474    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-6e81aea5\"/\"default-dockercfg-p2g8h\""
Oct 31 04:07:07.275846 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.275816    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-tmp-dir\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.276028 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.275858    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-model-cache\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.276028 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.275900    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-dshm\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.276028 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.275923    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t556\" (UniqueName: \"kubernetes.io/projected/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-kube-api-access-4t556\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.276028 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.275965    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-tls-certs\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.276254 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.276043    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-home\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.276254 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.276248    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-tmp-dir\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.276396 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.276306    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-model-cache\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.276460 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.276435    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-home\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.278632 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.278608    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-tls-certs\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.278753 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.278685    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-dshm\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.339307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.339252    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t556\" (UniqueName: \"kubernetes.io/projected/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-kube-api-access-4t556\") pod \"e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") " pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.446721 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.446623    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:07.828625 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:07:07.828594    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode430bd98_6cc2_4b3a_a521_71a3e2547fdc.slice/crio-88635d6e1bbad629ff47e45b7dd663c705f9bf4ab0e44c4c69b67972254c41d4 WatchSource:0}: Error finding container 88635d6e1bbad629ff47e45b7dd663c705f9bf4ab0e44c4c69b67972254c41d4: Status 404 returned error can't find the container with id 88635d6e1bbad629ff47e45b7dd663c705f9bf4ab0e44c4c69b67972254c41d4
Oct 31 04:07:07.837316 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:07.834812    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"]
Oct 31 04:07:08.659550 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:08.659512    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn" event={"ID":"e430bd98-6cc2-4b3a-a521-71a3e2547fdc","Type":"ContainerStarted","Data":"64eba575b8823370f88377a9eff06730c784bae28401f5c56589ced75af40cf5"}
Oct 31 04:07:08.659550 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:08.659554    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn" event={"ID":"e430bd98-6cc2-4b3a-a521-71a3e2547fdc","Type":"ContainerStarted","Data":"88635d6e1bbad629ff47e45b7dd663c705f9bf4ab0e44c4c69b67972254c41d4"}
Oct 31 04:07:08.780428 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:08.780374    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn" podStartSLOduration=1.780360889 podStartE2EDuration="1.780360889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:07:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:07:08.778165894 +0000 UTC m=+2325.363706155" watchObservedRunningTime="2026-10-31 04:07:08.780360889 +0000 UTC m=+2325.365901151"
Oct 31 04:07:09.447096 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:09.447063    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:13.845919 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:13.845881    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"]
Oct 31 04:07:13.846422 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:13.846094    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448" podUID="69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" containerName="istio-proxy" containerID="cri-o://cfc6fb2817a10e6fbdb17ef303e8976b303dbbf43af4c322e6f49086f5a8fdbd" gracePeriod=30
Oct 31 04:07:16.000204 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:16.000167    4926 patch_prober.go:29] interesting pod/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:07:16.000635 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:16.000234    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448" podUID="69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:07:17.447340 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:17.447309    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:17.460076 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:17.460044    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:17.721314 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:17.721286    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:07:19.111666 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.111610    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 04:07:19.229966 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.229935    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-data\") pod \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") "
Oct 31 04:07:19.230126 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.230006    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-podinfo\") pod \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") "
Oct 31 04:07:19.230126 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.230033    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istiod-ca-cert\") pod \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") "
Oct 31 04:07:19.230126 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.230103    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-kube-api-access-7w4r2\" (UniqueName: \"kubernetes.io/projected/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-kube-api-access-7w4r2\") pod \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") "
Oct 31 04:07:19.230292 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.230131    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-envoy\") pod \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") "
Oct 31 04:07:19.230292 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.230166    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-workload-socket\") pod \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") "
Oct 31 04:07:19.230292 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.230263    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-token\" (UniqueName: \"kubernetes.io/projected/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-token\") pod \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") "
Oct 31 04:07:19.230444 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.230258    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-data" pod "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" (UID: "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:07:19.230444 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.230330    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-credential-socket\") pod \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") "
Oct 31 04:07:19.230444 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.230369    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-workload-certs\") pod \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\" (UID: \"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0\") "
Oct 31 04:07:19.230595 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.230580    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-workload-socket" pod "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" (UID: "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:07:19.230696 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.230616    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-credential-socket" pod "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" (UID: "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:07:19.230696 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.230638    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istiod-ca-cert" pod "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" (UID: "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:07:19.230798 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.230732    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-workload-certs" pod "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" (UID: "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:07:19.230913 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.230889    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:19.231106 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.231083    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:19.231226 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.231109    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:19.231226 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.231126    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:19.231226 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.231178    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:19.232668 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.232646    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-podinfo" pod "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" (UID: "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:07:19.232813 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.232780    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-envoy" pod "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" (UID: "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:07:19.232910 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.232892    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-kube-api-access-7w4r2" pod "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" (UID: "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0"). InnerVolumeSpecName "kube-api-access-7w4r2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:07:19.232962 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.232923    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-token" pod "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" (UID: "69775e74-fc6d-4f0d-9f2e-b8e291ca00b0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:07:19.332630 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.332596    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:19.332878 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.332850    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w4r2\" (UniqueName: \"kubernetes.io/projected/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-kube-api-access-7w4r2\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:19.333006 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.332882    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:19.333006 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.332897    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:19.721344 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.721256    4926 generic.go:356] "Generic (PLEG): container finished" podID="69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" containerID="cfc6fb2817a10e6fbdb17ef303e8976b303dbbf43af4c322e6f49086f5a8fdbd" exitCode=0
Oct 31 04:07:19.721344 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.721304    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448" event={"ID":"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0","Type":"ContainerDied","Data":"cfc6fb2817a10e6fbdb17ef303e8976b303dbbf43af4c322e6f49086f5a8fdbd"}
Oct 31 04:07:19.721344 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.721336    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"
Oct 31 04:07:19.721609 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.721374    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448" event={"ID":"69775e74-fc6d-4f0d-9f2e-b8e291ca00b0","Type":"ContainerDied","Data":"daf7c6f50db010945ad5f3d0246b6cd48862d53e65fec4713a7d2ab669376631"}
Oct 31 04:07:19.721609 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.721405    4926 scope.go:122] "RemoveContainer" containerID="cfc6fb2817a10e6fbdb17ef303e8976b303dbbf43af4c322e6f49086f5a8fdbd"
Oct 31 04:07:19.736645 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.736597    4926 scope.go:122] "RemoveContainer" containerID="cfc6fb2817a10e6fbdb17ef303e8976b303dbbf43af4c322e6f49086f5a8fdbd"
Oct 31 04:07:19.737021 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:07:19.736994    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfc6fb2817a10e6fbdb17ef303e8976b303dbbf43af4c322e6f49086f5a8fdbd\": container with ID starting with cfc6fb2817a10e6fbdb17ef303e8976b303dbbf43af4c322e6f49086f5a8fdbd not found: ID does not exist" containerID="cfc6fb2817a10e6fbdb17ef303e8976b303dbbf43af4c322e6f49086f5a8fdbd"
Oct 31 04:07:19.737101 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.737033    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfc6fb2817a10e6fbdb17ef303e8976b303dbbf43af4c322e6f49086f5a8fdbd"} err="failed to get container status \"cfc6fb2817a10e6fbdb17ef303e8976b303dbbf43af4c322e6f49086f5a8fdbd\": rpc error: code = NotFound desc = could not find container \"cfc6fb2817a10e6fbdb17ef303e8976b303dbbf43af4c322e6f49086f5a8fdbd\": container with ID starting with cfc6fb2817a10e6fbdb17ef303e8976b303dbbf43af4c322e6f49086f5a8fdbd not found: ID does not exist"
Oct 31 04:07:19.891793 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.891713    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"]
Oct 31 04:07:19.943684 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:19.943654    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-412ba5-openshift-default-554dbd8599-pb448"]
Oct 31 04:07:21.538873 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:21.538839    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" path="/var/lib/kubelet/pods/69775e74-fc6d-4f0d-9f2e-b8e291ca00b0/volumes"
Oct 31 04:07:35.313875 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:07:35.313836    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:07:53.043848 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.043812    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"]
Oct 31 04:07:53.044428 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.044028    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk" podUID="0664091e-7353-433c-bd14-76c8b1b6d795" containerName="maas-api" containerID="cri-o://e98232df5eb9824bcbac8a2290064ce47dfb9687da454f130920f731fad453b5" gracePeriod=30
Oct 31 04:07:53.320862 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.320829    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 04:07:53.422546 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.422515    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-maas-api-tls\") pod \"0664091e-7353-433c-bd14-76c8b1b6d795\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") "
Oct 31 04:07:53.422744 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.422600    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-metrics-certs\") pod \"0664091e-7353-433c-bd14-76c8b1b6d795\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") "
Oct 31 04:07:53.422744 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.422670    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0664091e-7353-433c-bd14-76c8b1b6d795-kube-api-access-kfcnq\" (UniqueName: \"kubernetes.io/projected/0664091e-7353-433c-bd14-76c8b1b6d795-kube-api-access-kfcnq\") pod \"0664091e-7353-433c-bd14-76c8b1b6d795\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") "
Oct 31 04:07:53.422744 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.422693    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0664091e-7353-433c-bd14-76c8b1b6d795-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0664091e-7353-433c-bd14-76c8b1b6d795-openshift-service-ca\") pod \"0664091e-7353-433c-bd14-76c8b1b6d795\" (UID: \"0664091e-7353-433c-bd14-76c8b1b6d795\") "
Oct 31 04:07:53.423196 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.423166    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0664091e-7353-433c-bd14-76c8b1b6d795-openshift-service-ca" pod "0664091e-7353-433c-bd14-76c8b1b6d795" (UID: "0664091e-7353-433c-bd14-76c8b1b6d795"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:07:53.424878 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.424849    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-metrics-certs" pod "0664091e-7353-433c-bd14-76c8b1b6d795" (UID: "0664091e-7353-433c-bd14-76c8b1b6d795"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:07:53.424970 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.424885    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-maas-api-tls" pod "0664091e-7353-433c-bd14-76c8b1b6d795" (UID: "0664091e-7353-433c-bd14-76c8b1b6d795"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:07:53.424970 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.424928    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0664091e-7353-433c-bd14-76c8b1b6d795-kube-api-access-kfcnq" pod "0664091e-7353-433c-bd14-76c8b1b6d795" (UID: "0664091e-7353-433c-bd14-76c8b1b6d795"). InnerVolumeSpecName "kube-api-access-kfcnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:07:53.525610 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.525503    4926 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-maas-api-tls\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:53.525610 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.525561    4926 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0664091e-7353-433c-bd14-76c8b1b6d795-metrics-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:53.526014 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.525577    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfcnq\" (UniqueName: \"kubernetes.io/projected/0664091e-7353-433c-bd14-76c8b1b6d795-kube-api-access-kfcnq\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:53.527908 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.527855    4926 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0664091e-7353-433c-bd14-76c8b1b6d795-openshift-service-ca\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:07:53.924350 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.924190    4926 generic.go:356] "Generic (PLEG): container finished" podID="0664091e-7353-433c-bd14-76c8b1b6d795" containerID="e98232df5eb9824bcbac8a2290064ce47dfb9687da454f130920f731fad453b5" exitCode=0
Oct 31 04:07:53.924350 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.924243    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk" event={"ID":"0664091e-7353-433c-bd14-76c8b1b6d795","Type":"ContainerDied","Data":"e98232df5eb9824bcbac8a2290064ce47dfb9687da454f130920f731fad453b5"}
Oct 31 04:07:53.924350 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.924316    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk" event={"ID":"0664091e-7353-433c-bd14-76c8b1b6d795","Type":"ContainerDied","Data":"8c5d1f3ea28c477e25bcbbc96d1b8a1a493aac356a15ae4cf2c5a685280705ea"}
Oct 31 04:07:53.924350 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.924350    4926 scope.go:122] "RemoveContainer" containerID="e98232df5eb9824bcbac8a2290064ce47dfb9687da454f130920f731fad453b5"
Oct 31 04:07:53.924785 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.924573    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"
Oct 31 04:07:53.946666 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.946623    4926 scope.go:122] "RemoveContainer" containerID="e98232df5eb9824bcbac8a2290064ce47dfb9687da454f130920f731fad453b5"
Oct 31 04:07:53.955288 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:07:53.953410    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e98232df5eb9824bcbac8a2290064ce47dfb9687da454f130920f731fad453b5\": container with ID starting with e98232df5eb9824bcbac8a2290064ce47dfb9687da454f130920f731fad453b5 not found: ID does not exist" containerID="e98232df5eb9824bcbac8a2290064ce47dfb9687da454f130920f731fad453b5"
Oct 31 04:07:53.955288 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:53.953458    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e98232df5eb9824bcbac8a2290064ce47dfb9687da454f130920f731fad453b5"} err="failed to get container status \"e98232df5eb9824bcbac8a2290064ce47dfb9687da454f130920f731fad453b5\": rpc error: code = NotFound desc = could not find container \"e98232df5eb9824bcbac8a2290064ce47dfb9687da454f130920f731fad453b5\": container with ID starting with e98232df5eb9824bcbac8a2290064ce47dfb9687da454f130920f731fad453b5 not found: ID does not exist"
Oct 31 04:07:54.094361 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:54.094326    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"]
Oct 31 04:07:54.101277 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:54.101240    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-d7b7bc-5dbf5c58fc-8f4lk"]
Oct 31 04:07:55.538639 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:07:55.538598    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0664091e-7353-433c-bd14-76c8b1b6d795" path="/var/lib/kubelet/pods/0664091e-7353-433c-bd14-76c8b1b6d795/volumes"
Oct 31 04:08:03.068698 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:03.068616    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"]
Oct 31 04:08:03.069108 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:03.068811    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8" podUID="7ce8937f-8f05-4e63-85bb-5ee282b5367b" containerName="istio-proxy" containerID="cri-o://c1ff09a2d939d10285ec82248550f64742fc22d533c86f343f37e118a58ae16c" gracePeriod=30
Oct 31 04:08:05.313423 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:08:05.313385    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:08:08.333477 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.333418    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 04:08:08.519023 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.518993    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-podinfo\") pod \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") "
Oct 31 04:08:08.519207 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519055    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-data\") pod \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") "
Oct 31 04:08:08.519207 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519077    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-credential-socket\") pod \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") "
Oct 31 04:08:08.519207 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519129    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istiod-ca-cert\") pod \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") "
Oct 31 04:08:08.519207 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519169    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-workload-certs\") pod \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") "
Oct 31 04:08:08.519207 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519200    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-workload-socket\") pod \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") "
Oct 31 04:08:08.519516 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519239    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-envoy\") pod \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") "
Oct 31 04:08:08.519516 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519326    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-data" pod "7ce8937f-8f05-4e63-85bb-5ee282b5367b" (UID: "7ce8937f-8f05-4e63-85bb-5ee282b5367b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:08.519516 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519336    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ce8937f-8f05-4e63-85bb-5ee282b5367b-kube-api-access-mj7h8\" (UniqueName: \"kubernetes.io/projected/7ce8937f-8f05-4e63-85bb-5ee282b5367b-kube-api-access-mj7h8\") pod \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") "
Oct 31 04:08:08.519516 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519458    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-token\" (UniqueName: \"kubernetes.io/projected/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-token\") pod \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\" (UID: \"7ce8937f-8f05-4e63-85bb-5ee282b5367b\") "
Oct 31 04:08:08.519516 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519496    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-workload-certs" pod "7ce8937f-8f05-4e63-85bb-5ee282b5367b" (UID: "7ce8937f-8f05-4e63-85bb-5ee282b5367b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:08.519780 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519517    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-credential-socket" pod "7ce8937f-8f05-4e63-85bb-5ee282b5367b" (UID: "7ce8937f-8f05-4e63-85bb-5ee282b5367b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:08.519780 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519543    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-workload-socket" pod "7ce8937f-8f05-4e63-85bb-5ee282b5367b" (UID: "7ce8937f-8f05-4e63-85bb-5ee282b5367b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:08.519780 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519559    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istiod-ca-cert" pod "7ce8937f-8f05-4e63-85bb-5ee282b5367b" (UID: "7ce8937f-8f05-4e63-85bb-5ee282b5367b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:08:08.519954 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519937    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:08.520090 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519959    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:08.520090 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519973    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:08.520090 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.519987    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:08.520090 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.520000    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:08.521874 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.521849    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-podinfo" pod "7ce8937f-8f05-4e63-85bb-5ee282b5367b" (UID: "7ce8937f-8f05-4e63-85bb-5ee282b5367b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:08:08.522166 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.522146    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-envoy" pod "7ce8937f-8f05-4e63-85bb-5ee282b5367b" (UID: "7ce8937f-8f05-4e63-85bb-5ee282b5367b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:08.522507 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.522483    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce8937f-8f05-4e63-85bb-5ee282b5367b-kube-api-access-mj7h8" pod "7ce8937f-8f05-4e63-85bb-5ee282b5367b" (UID: "7ce8937f-8f05-4e63-85bb-5ee282b5367b"). InnerVolumeSpecName "kube-api-access-mj7h8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:08:08.522593 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.522503    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-token" pod "7ce8937f-8f05-4e63-85bb-5ee282b5367b" (UID: "7ce8937f-8f05-4e63-85bb-5ee282b5367b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:08:08.620834 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.620760    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:08.620834 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.620786    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:08.620834 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.620795    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce8937f-8f05-4e63-85bb-5ee282b5367b-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:08.620834 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:08.620804    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj7h8\" (UniqueName: \"kubernetes.io/projected/7ce8937f-8f05-4e63-85bb-5ee282b5367b-kube-api-access-mj7h8\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:09.002593 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:09.002552    4926 generic.go:356] "Generic (PLEG): container finished" podID="7ce8937f-8f05-4e63-85bb-5ee282b5367b" containerID="c1ff09a2d939d10285ec82248550f64742fc22d533c86f343f37e118a58ae16c" exitCode=0
Oct 31 04:08:09.002793 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:09.002591    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8" event={"ID":"7ce8937f-8f05-4e63-85bb-5ee282b5367b","Type":"ContainerDied","Data":"c1ff09a2d939d10285ec82248550f64742fc22d533c86f343f37e118a58ae16c"}
Oct 31 04:08:09.002793 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:09.002631    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"
Oct 31 04:08:09.002793 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:09.002673    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8" event={"ID":"7ce8937f-8f05-4e63-85bb-5ee282b5367b","Type":"ContainerDied","Data":"c2b47f56b149f7c2c9f38e4bfc3f556f75af6611a4d191b7fd7219c4f69dd146"}
Oct 31 04:08:09.002793 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:09.002701    4926 scope.go:122] "RemoveContainer" containerID="c1ff09a2d939d10285ec82248550f64742fc22d533c86f343f37e118a58ae16c"
Oct 31 04:08:09.012510 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:09.012494    4926 scope.go:122] "RemoveContainer" containerID="8681d42d6bb99b5db00bef4f8a5917b3f14aa67df0ec6ea0c9e93ac0ed4fc6bf"
Oct 31 04:08:09.023253 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:09.023231    4926 scope.go:122] "RemoveContainer" containerID="c1ff09a2d939d10285ec82248550f64742fc22d533c86f343f37e118a58ae16c"
Oct 31 04:08:09.023694 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:08:09.023669    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1ff09a2d939d10285ec82248550f64742fc22d533c86f343f37e118a58ae16c\": container with ID starting with c1ff09a2d939d10285ec82248550f64742fc22d533c86f343f37e118a58ae16c not found: ID does not exist" containerID="c1ff09a2d939d10285ec82248550f64742fc22d533c86f343f37e118a58ae16c"
Oct 31 04:08:09.023789 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:09.023704    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1ff09a2d939d10285ec82248550f64742fc22d533c86f343f37e118a58ae16c"} err="failed to get container status \"c1ff09a2d939d10285ec82248550f64742fc22d533c86f343f37e118a58ae16c\": rpc error: code = NotFound desc = could not find container \"c1ff09a2d939d10285ec82248550f64742fc22d533c86f343f37e118a58ae16c\": container with ID starting with c1ff09a2d939d10285ec82248550f64742fc22d533c86f343f37e118a58ae16c not found: ID does not exist"
Oct 31 04:08:09.023789 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:09.023728    4926 scope.go:122] "RemoveContainer" containerID="8681d42d6bb99b5db00bef4f8a5917b3f14aa67df0ec6ea0c9e93ac0ed4fc6bf"
Oct 31 04:08:09.024024 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:08:09.024003    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8681d42d6bb99b5db00bef4f8a5917b3f14aa67df0ec6ea0c9e93ac0ed4fc6bf\": container with ID starting with 8681d42d6bb99b5db00bef4f8a5917b3f14aa67df0ec6ea0c9e93ac0ed4fc6bf not found: ID does not exist" containerID="8681d42d6bb99b5db00bef4f8a5917b3f14aa67df0ec6ea0c9e93ac0ed4fc6bf"
Oct 31 04:08:09.024078 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:09.024031    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8681d42d6bb99b5db00bef4f8a5917b3f14aa67df0ec6ea0c9e93ac0ed4fc6bf"} err="failed to get container status \"8681d42d6bb99b5db00bef4f8a5917b3f14aa67df0ec6ea0c9e93ac0ed4fc6bf\": rpc error: code = NotFound desc = could not find container \"8681d42d6bb99b5db00bef4f8a5917b3f14aa67df0ec6ea0c9e93ac0ed4fc6bf\": container with ID starting with 8681d42d6bb99b5db00bef4f8a5917b3f14aa67df0ec6ea0c9e93ac0ed4fc6bf not found: ID does not exist"
Oct 31 04:08:09.171837 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:09.171806    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"]
Oct 31 04:08:09.184276 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:09.184244    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-d7b7bc-openshift-default-5b48c74cfc-m7wv8"]
Oct 31 04:08:09.538849 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:09.538818    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ce8937f-8f05-4e63-85bb-5ee282b5367b" path="/var/lib/kubelet/pods/7ce8937f-8f05-4e63-85bb-5ee282b5367b/volumes"
Oct 31 04:08:23.804756 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:23.804730    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:08:23.808523 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:23.808502    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:08:30.416717 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:30.416682    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"]
Oct 31 04:08:30.417530 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:30.417473    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5" podUID="a977c9a1-fe79-4f13-8aa0-9a3484a28238" containerName="istio-proxy" containerID="cri-o://4f46e0ba4ab20e1fe4cc3f07ee82c9f9c858fb5edbc39ccb553b5562d29245b5" gracePeriod=30
Oct 31 04:08:30.431728 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:30.431701    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"]
Oct 31 04:08:30.431910 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:30.431891    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" podUID="a9f9a022-fa1c-4f08-ac54-cb751f629547" containerName="istio-proxy" containerID="cri-o://1c5b41c24dc4419305a9902fd7d8c402b3c7ddc9f4b0594ed678e4338247fb17" gracePeriod=30
Oct 31 04:08:31.357985 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:31.357950    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:08:31.358156 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:31.358005    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5" podUID="a977c9a1-fe79-4f13-8aa0-9a3484a28238" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:08:33.465365 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:33.465326    4926 patch_prober.go:29] interesting pod/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:08:33.465745 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:33.465393    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" podUID="a9f9a022-fa1c-4f08-ac54-cb751f629547" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:08:35.316655 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:08:35.316614    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:08:35.685395 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.685371    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 04:08:35.710738 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.710711    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 04:08:35.824990 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.824961    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-workload-certs\") pod \"a9f9a022-fa1c-4f08-ac54-cb751f629547\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") "
Oct 31 04:08:35.825149 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825025    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-envoy\") pod \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") "
Oct 31 04:08:35.825149 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825061    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-data\") pod \"a9f9a022-fa1c-4f08-ac54-cb751f629547\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") "
Oct 31 04:08:35.825149 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825080    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-envoy\") pod \"a9f9a022-fa1c-4f08-ac54-cb751f629547\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") "
Oct 31 04:08:35.825149 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825120    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-token\" (UniqueName: \"kubernetes.io/projected/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-token\") pod \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") "
Oct 31 04:08:35.825410 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825155    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a977c9a1-fe79-4f13-8aa0-9a3484a28238-kube-api-access-tj5fb\" (UniqueName: \"kubernetes.io/projected/a977c9a1-fe79-4f13-8aa0-9a3484a28238-kube-api-access-tj5fb\") pod \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") "
Oct 31 04:08:35.825410 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825200    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-credential-socket\") pod \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") "
Oct 31 04:08:35.825410 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825227    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-workload-socket\") pod \"a9f9a022-fa1c-4f08-ac54-cb751f629547\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") "
Oct 31 04:08:35.825410 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825256    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-data\") pod \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") "
Oct 31 04:08:35.825410 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825309    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-workload-certs\") pod \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") "
Oct 31 04:08:35.825410 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825334    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-workload-certs" pod "a9f9a022-fa1c-4f08-ac54-cb751f629547" (UID: "a9f9a022-fa1c-4f08-ac54-cb751f629547"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:35.825410 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825342    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-token\" (UniqueName: \"kubernetes.io/projected/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-token\") pod \"a9f9a022-fa1c-4f08-ac54-cb751f629547\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") "
Oct 31 04:08:35.825410 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825399    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-data" pod "a9f9a022-fa1c-4f08-ac54-cb751f629547" (UID: "a9f9a022-fa1c-4f08-ac54-cb751f629547"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:35.825837 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825421    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-podinfo\") pod \"a9f9a022-fa1c-4f08-ac54-cb751f629547\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") "
Oct 31 04:08:35.825837 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825478    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-workload-socket\") pod \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") "
Oct 31 04:08:35.825837 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825553    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-podinfo\") pod \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") "
Oct 31 04:08:35.825837 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825590    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-credential-socket\") pod \"a9f9a022-fa1c-4f08-ac54-cb751f629547\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") "
Oct 31 04:08:35.825837 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825641    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9f9a022-fa1c-4f08-ac54-cb751f629547-kube-api-access-95svf\" (UniqueName: \"kubernetes.io/projected/a9f9a022-fa1c-4f08-ac54-cb751f629547-kube-api-access-95svf\") pod \"a9f9a022-fa1c-4f08-ac54-cb751f629547\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") "
Oct 31 04:08:35.825837 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825705    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istiod-ca-cert\") pod \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\" (UID: \"a977c9a1-fe79-4f13-8aa0-9a3484a28238\") "
Oct 31 04:08:35.825837 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825748    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9f9a022-fa1c-4f08-ac54-cb751f629547-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9f9a022-fa1c-4f08-ac54-cb751f629547-istiod-ca-cert\") pod \"a9f9a022-fa1c-4f08-ac54-cb751f629547\" (UID: \"a9f9a022-fa1c-4f08-ac54-cb751f629547\") "
Oct 31 04:08:35.825837 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.825767    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-data" pod "a977c9a1-fe79-4f13-8aa0-9a3484a28238" (UID: "a977c9a1-fe79-4f13-8aa0-9a3484a28238"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:35.826258 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.826127    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-workload-socket" pod "a9f9a022-fa1c-4f08-ac54-cb751f629547" (UID: "a9f9a022-fa1c-4f08-ac54-cb751f629547"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:35.826258 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.826169    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-credential-socket" pod "a977c9a1-fe79-4f13-8aa0-9a3484a28238" (UID: "a977c9a1-fe79-4f13-8aa0-9a3484a28238"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:35.826451 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.826427    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.826623 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.826594    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.826743 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.826727    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.826842 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.826829    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.826926 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.826914    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.827004 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.826635    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istiod-ca-cert" pod "a977c9a1-fe79-4f13-8aa0-9a3484a28238" (UID: "a977c9a1-fe79-4f13-8aa0-9a3484a28238"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:08:35.827088 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.826786    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-workload-socket" pod "a977c9a1-fe79-4f13-8aa0-9a3484a28238" (UID: "a977c9a1-fe79-4f13-8aa0-9a3484a28238"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:35.827167 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.826711    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9f9a022-fa1c-4f08-ac54-cb751f629547-istiod-ca-cert" pod "a9f9a022-fa1c-4f08-ac54-cb751f629547" (UID: "a9f9a022-fa1c-4f08-ac54-cb751f629547"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:08:35.827303 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.827106    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-workload-certs" pod "a977c9a1-fe79-4f13-8aa0-9a3484a28238" (UID: "a977c9a1-fe79-4f13-8aa0-9a3484a28238"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:35.827421 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.827361    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-credential-socket" pod "a9f9a022-fa1c-4f08-ac54-cb751f629547" (UID: "a9f9a022-fa1c-4f08-ac54-cb751f629547"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:35.829055 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.829021    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-envoy" pod "a9f9a022-fa1c-4f08-ac54-cb751f629547" (UID: "a9f9a022-fa1c-4f08-ac54-cb751f629547"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:35.829055 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.829040    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-token" pod "a9f9a022-fa1c-4f08-ac54-cb751f629547" (UID: "a9f9a022-fa1c-4f08-ac54-cb751f629547"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:08:35.829352 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.829317    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a977c9a1-fe79-4f13-8aa0-9a3484a28238-kube-api-access-tj5fb" pod "a977c9a1-fe79-4f13-8aa0-9a3484a28238" (UID: "a977c9a1-fe79-4f13-8aa0-9a3484a28238"). InnerVolumeSpecName "kube-api-access-tj5fb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:08:35.829649 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.829622    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-podinfo" pod "a9f9a022-fa1c-4f08-ac54-cb751f629547" (UID: "a9f9a022-fa1c-4f08-ac54-cb751f629547"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:08:35.829952 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.829929    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-token" pod "a977c9a1-fe79-4f13-8aa0-9a3484a28238" (UID: "a977c9a1-fe79-4f13-8aa0-9a3484a28238"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:08:35.830052 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.830017    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-envoy" pod "a977c9a1-fe79-4f13-8aa0-9a3484a28238" (UID: "a977c9a1-fe79-4f13-8aa0-9a3484a28238"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:35.830561 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.830540    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-podinfo" pod "a977c9a1-fe79-4f13-8aa0-9a3484a28238" (UID: "a977c9a1-fe79-4f13-8aa0-9a3484a28238"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:08:35.830891 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.830876    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9f9a022-fa1c-4f08-ac54-cb751f629547-kube-api-access-95svf" pod "a9f9a022-fa1c-4f08-ac54-cb751f629547" (UID: "a9f9a022-fa1c-4f08-ac54-cb751f629547"). InnerVolumeSpecName "kube-api-access-95svf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:08:35.927886 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.927800    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.927886 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.927831    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.927886 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.927844    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.927886 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.927855    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.927886 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.927867    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.927886 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.927880    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95svf\" (UniqueName: \"kubernetes.io/projected/a9f9a022-fa1c-4f08-ac54-cb751f629547-kube-api-access-95svf\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.928199 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.927896    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.928199 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.927909    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9f9a022-fa1c-4f08-ac54-cb751f629547-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.928199 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.927920    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.928199 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.927931    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9f9a022-fa1c-4f08-ac54-cb751f629547-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.928199 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.927942    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a977c9a1-fe79-4f13-8aa0-9a3484a28238-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.928199 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.927955    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj5fb\" (UniqueName: \"kubernetes.io/projected/a977c9a1-fe79-4f13-8aa0-9a3484a28238-kube-api-access-tj5fb\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:35.928199 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:35.927967    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a977c9a1-fe79-4f13-8aa0-9a3484a28238-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:36.150907 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.150857    4926 generic.go:356] "Generic (PLEG): container finished" podID="a9f9a022-fa1c-4f08-ac54-cb751f629547" containerID="1c5b41c24dc4419305a9902fd7d8c402b3c7ddc9f4b0594ed678e4338247fb17" exitCode=0
Oct 31 04:08:36.150907 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.150904    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"
Oct 31 04:08:36.151166 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.150948    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" event={"ID":"a9f9a022-fa1c-4f08-ac54-cb751f629547","Type":"ContainerDied","Data":"1c5b41c24dc4419305a9902fd7d8c402b3c7ddc9f4b0594ed678e4338247fb17"}
Oct 31 04:08:36.151166 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.151002    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz" event={"ID":"a9f9a022-fa1c-4f08-ac54-cb751f629547","Type":"ContainerDied","Data":"84e80ad0822c95ad77aa867e8e4f90cf443d6c180cca7a7933bca9570657db78"}
Oct 31 04:08:36.151166 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.151023    4926 scope.go:122] "RemoveContainer" containerID="1c5b41c24dc4419305a9902fd7d8c402b3c7ddc9f4b0594ed678e4338247fb17"
Oct 31 04:08:36.152464 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.152438    4926 generic.go:356] "Generic (PLEG): container finished" podID="a977c9a1-fe79-4f13-8aa0-9a3484a28238" containerID="4f46e0ba4ab20e1fe4cc3f07ee82c9f9c858fb5edbc39ccb553b5562d29245b5" exitCode=0
Oct 31 04:08:36.152792 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.152491    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"
Oct 31 04:08:36.152792 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.152486    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5" event={"ID":"a977c9a1-fe79-4f13-8aa0-9a3484a28238","Type":"ContainerDied","Data":"4f46e0ba4ab20e1fe4cc3f07ee82c9f9c858fb5edbc39ccb553b5562d29245b5"}
Oct 31 04:08:36.152792 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.152610    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5" event={"ID":"a977c9a1-fe79-4f13-8aa0-9a3484a28238","Type":"ContainerDied","Data":"22e043e78ac64829e73c6de8a043b081790d719d942160d2d79acb28799dfb89"}
Oct 31 04:08:36.160562 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.160542    4926 scope.go:122] "RemoveContainer" containerID="1c5b41c24dc4419305a9902fd7d8c402b3c7ddc9f4b0594ed678e4338247fb17"
Oct 31 04:08:36.160883 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:08:36.160865    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c5b41c24dc4419305a9902fd7d8c402b3c7ddc9f4b0594ed678e4338247fb17\": container with ID starting with 1c5b41c24dc4419305a9902fd7d8c402b3c7ddc9f4b0594ed678e4338247fb17 not found: ID does not exist" containerID="1c5b41c24dc4419305a9902fd7d8c402b3c7ddc9f4b0594ed678e4338247fb17"
Oct 31 04:08:36.160930 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.160892    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c5b41c24dc4419305a9902fd7d8c402b3c7ddc9f4b0594ed678e4338247fb17"} err="failed to get container status \"1c5b41c24dc4419305a9902fd7d8c402b3c7ddc9f4b0594ed678e4338247fb17\": rpc error: code = NotFound desc = could not find container \"1c5b41c24dc4419305a9902fd7d8c402b3c7ddc9f4b0594ed678e4338247fb17\": container with ID starting with 1c5b41c24dc4419305a9902fd7d8c402b3c7ddc9f4b0594ed678e4338247fb17 not found: ID does not exist"
Oct 31 04:08:36.160930 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.160913    4926 scope.go:122] "RemoveContainer" containerID="4f46e0ba4ab20e1fe4cc3f07ee82c9f9c858fb5edbc39ccb553b5562d29245b5"
Oct 31 04:08:36.170110 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.170085    4926 scope.go:122] "RemoveContainer" containerID="4f46e0ba4ab20e1fe4cc3f07ee82c9f9c858fb5edbc39ccb553b5562d29245b5"
Oct 31 04:08:36.170408 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:08:36.170390    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f46e0ba4ab20e1fe4cc3f07ee82c9f9c858fb5edbc39ccb553b5562d29245b5\": container with ID starting with 4f46e0ba4ab20e1fe4cc3f07ee82c9f9c858fb5edbc39ccb553b5562d29245b5 not found: ID does not exist" containerID="4f46e0ba4ab20e1fe4cc3f07ee82c9f9c858fb5edbc39ccb553b5562d29245b5"
Oct 31 04:08:36.170474 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.170414    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f46e0ba4ab20e1fe4cc3f07ee82c9f9c858fb5edbc39ccb553b5562d29245b5"} err="failed to get container status \"4f46e0ba4ab20e1fe4cc3f07ee82c9f9c858fb5edbc39ccb553b5562d29245b5\": rpc error: code = NotFound desc = could not find container \"4f46e0ba4ab20e1fe4cc3f07ee82c9f9c858fb5edbc39ccb553b5562d29245b5\": container with ID starting with 4f46e0ba4ab20e1fe4cc3f07ee82c9f9c858fb5edbc39ccb553b5562d29245b5 not found: ID does not exist"
Oct 31 04:08:36.321285 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.321228    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"]
Oct 31 04:08:36.324916 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.324886    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-vjsl5"]
Oct 31 04:08:36.443404 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.443368    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"]
Oct 31 04:08:36.446692 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:36.446649    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-537e34-openshift-default-57bf7cc76f-cn5cz"]
Oct 31 04:08:37.546825 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:37.546784    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a977c9a1-fe79-4f13-8aa0-9a3484a28238" path="/var/lib/kubelet/pods/a977c9a1-fe79-4f13-8aa0-9a3484a28238/volumes"
Oct 31 04:08:37.547445 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:37.547423    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9f9a022-fa1c-4f08-ac54-cb751f629547" path="/var/lib/kubelet/pods/a9f9a022-fa1c-4f08-ac54-cb751f629547/volumes"
Oct 31 04:08:51.666978 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:51.666932    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"]
Oct 31 04:08:51.667462 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:51.667133    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz" podUID="a65ad9de-668e-4600-8643-546a6ce94b99" containerName="istio-proxy" containerID="cri-o://b31caaadab9c9cb7d86cd705e93bdead0a8a4ec3ccf7d9b4a71c806d0c85837d" gracePeriod=30
Oct 31 04:08:56.933817 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:56.933761    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 04:08:57.069914 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.069867    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-credential-socket\") pod \"a65ad9de-668e-4600-8643-546a6ce94b99\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") "
Oct 31 04:08:57.070100 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.069926    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a65ad9de-668e-4600-8643-546a6ce94b99-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a65ad9de-668e-4600-8643-546a6ce94b99-istio-podinfo\") pod \"a65ad9de-668e-4600-8643-546a6ce94b99\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") "
Oct 31 04:08:57.070100 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.069975    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a65ad9de-668e-4600-8643-546a6ce94b99-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a65ad9de-668e-4600-8643-546a6ce94b99-istiod-ca-cert\") pod \"a65ad9de-668e-4600-8643-546a6ce94b99\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") "
Oct 31 04:08:57.070100 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.070049    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a65ad9de-668e-4600-8643-546a6ce94b99-kube-api-access-mxmvl\" (UniqueName: \"kubernetes.io/projected/a65ad9de-668e-4600-8643-546a6ce94b99-kube-api-access-mxmvl\") pod \"a65ad9de-668e-4600-8643-546a6ce94b99\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") "
Oct 31 04:08:57.070325 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.070105    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-istio-envoy\") pod \"a65ad9de-668e-4600-8643-546a6ce94b99\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") "
Oct 31 04:08:57.070325 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.070162    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-istio-data\") pod \"a65ad9de-668e-4600-8643-546a6ce94b99\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") "
Oct 31 04:08:57.070325 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.070180    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-credential-socket" pod "a65ad9de-668e-4600-8643-546a6ce94b99" (UID: "a65ad9de-668e-4600-8643-546a6ce94b99"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:57.070325 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.070226    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-workload-socket\") pod \"a65ad9de-668e-4600-8643-546a6ce94b99\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") "
Oct 31 04:08:57.070325 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.070297    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-workload-certs\") pod \"a65ad9de-668e-4600-8643-546a6ce94b99\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") "
Oct 31 04:08:57.070652 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.070353    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a65ad9de-668e-4600-8643-546a6ce94b99-istio-token\" (UniqueName: \"kubernetes.io/projected/a65ad9de-668e-4600-8643-546a6ce94b99-istio-token\") pod \"a65ad9de-668e-4600-8643-546a6ce94b99\" (UID: \"a65ad9de-668e-4600-8643-546a6ce94b99\") "
Oct 31 04:08:57.070710 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.070684    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a65ad9de-668e-4600-8643-546a6ce94b99-istiod-ca-cert" pod "a65ad9de-668e-4600-8643-546a6ce94b99" (UID: "a65ad9de-668e-4600-8643-546a6ce94b99"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:08:57.071194 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.070867    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-istio-data" pod "a65ad9de-668e-4600-8643-546a6ce94b99" (UID: "a65ad9de-668e-4600-8643-546a6ce94b99"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:57.071194 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.070924    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:57.071194 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.070945    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a65ad9de-668e-4600-8643-546a6ce94b99-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:57.071194 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.071067    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-workload-certs" pod "a65ad9de-668e-4600-8643-546a6ce94b99" (UID: "a65ad9de-668e-4600-8643-546a6ce94b99"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:57.071194 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.071158    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-workload-socket" pod "a65ad9de-668e-4600-8643-546a6ce94b99" (UID: "a65ad9de-668e-4600-8643-546a6ce94b99"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:57.073626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.073600    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a65ad9de-668e-4600-8643-546a6ce94b99-kube-api-access-mxmvl" pod "a65ad9de-668e-4600-8643-546a6ce94b99" (UID: "a65ad9de-668e-4600-8643-546a6ce94b99"). InnerVolumeSpecName "kube-api-access-mxmvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:08:57.076460 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.076438    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a65ad9de-668e-4600-8643-546a6ce94b99-istio-token" pod "a65ad9de-668e-4600-8643-546a6ce94b99" (UID: "a65ad9de-668e-4600-8643-546a6ce94b99"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:08:57.076460 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.076443    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a65ad9de-668e-4600-8643-546a6ce94b99-istio-podinfo" pod "a65ad9de-668e-4600-8643-546a6ce94b99" (UID: "a65ad9de-668e-4600-8643-546a6ce94b99"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:08:57.076617 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.076549    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-istio-envoy" pod "a65ad9de-668e-4600-8643-546a6ce94b99" (UID: "a65ad9de-668e-4600-8643-546a6ce94b99"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:08:57.172446 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.172406    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:57.172446 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.172441    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a65ad9de-668e-4600-8643-546a6ce94b99-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:57.172706 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.172456    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a65ad9de-668e-4600-8643-546a6ce94b99-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:57.172706 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.172470    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxmvl\" (UniqueName: \"kubernetes.io/projected/a65ad9de-668e-4600-8643-546a6ce94b99-kube-api-access-mxmvl\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:57.172706 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.172484    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:57.172706 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.172497    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:57.172706 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.172510    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a65ad9de-668e-4600-8643-546a6ce94b99-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:08:57.263352 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.263317    4926 generic.go:356] "Generic (PLEG): container finished" podID="a65ad9de-668e-4600-8643-546a6ce94b99" containerID="b31caaadab9c9cb7d86cd705e93bdead0a8a4ec3ccf7d9b4a71c806d0c85837d" exitCode=0
Oct 31 04:08:57.263540 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.263370    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"
Oct 31 04:08:57.263540 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.263408    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz" event={"ID":"a65ad9de-668e-4600-8643-546a6ce94b99","Type":"ContainerDied","Data":"b31caaadab9c9cb7d86cd705e93bdead0a8a4ec3ccf7d9b4a71c806d0c85837d"}
Oct 31 04:08:57.263540 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.263460    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz" event={"ID":"a65ad9de-668e-4600-8643-546a6ce94b99","Type":"ContainerDied","Data":"3a430a2ccba9d6d78b03d9eeb062abee83364abef27360694ecdf3949c8e8a79"}
Oct 31 04:08:57.263540 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.263482    4926 scope.go:122] "RemoveContainer" containerID="b31caaadab9c9cb7d86cd705e93bdead0a8a4ec3ccf7d9b4a71c806d0c85837d"
Oct 31 04:08:57.273330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.273309    4926 scope.go:122] "RemoveContainer" containerID="b31caaadab9c9cb7d86cd705e93bdead0a8a4ec3ccf7d9b4a71c806d0c85837d"
Oct 31 04:08:57.273601 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:08:57.273581    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b31caaadab9c9cb7d86cd705e93bdead0a8a4ec3ccf7d9b4a71c806d0c85837d\": container with ID starting with b31caaadab9c9cb7d86cd705e93bdead0a8a4ec3ccf7d9b4a71c806d0c85837d not found: ID does not exist" containerID="b31caaadab9c9cb7d86cd705e93bdead0a8a4ec3ccf7d9b4a71c806d0c85837d"
Oct 31 04:08:57.273667 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.273610    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b31caaadab9c9cb7d86cd705e93bdead0a8a4ec3ccf7d9b4a71c806d0c85837d"} err="failed to get container status \"b31caaadab9c9cb7d86cd705e93bdead0a8a4ec3ccf7d9b4a71c806d0c85837d\": rpc error: code = NotFound desc = could not find container \"b31caaadab9c9cb7d86cd705e93bdead0a8a4ec3ccf7d9b4a71c806d0c85837d\": container with ID starting with b31caaadab9c9cb7d86cd705e93bdead0a8a4ec3ccf7d9b4a71c806d0c85837d not found: ID does not exist"
Oct 31 04:08:57.433637 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.433601    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"]
Oct 31 04:08:57.438737 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.438710    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-a64fd2-openshift-default-694cc85679-rzzwz"]
Oct 31 04:08:57.538216 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:08:57.538126    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a65ad9de-668e-4600-8643-546a6ce94b99" path="/var/lib/kubelet/pods/a65ad9de-668e-4600-8643-546a6ce94b99/volumes"
Oct 31 04:09:05.313950 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:09:05.313910    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:09:30.979384 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:30.979342    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"]
Oct 31 04:09:30.979970 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:30.979679    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv" podUID="a07d7017-aef8-49fb-8de0-0b3cba42c240" containerName="maas-api" containerID="cri-o://8ef6ee32cff2ffa1f047256fc76443df5b460280cefc7eb07207d45b8fe32aa2" gracePeriod=30
Oct 31 04:09:31.254346 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.254314    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:09:31.340387 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.340348    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a07d7017-aef8-49fb-8de0-0b3cba42c240-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07d7017-aef8-49fb-8de0-0b3cba42c240-openshift-service-ca\") pod \"a07d7017-aef8-49fb-8de0-0b3cba42c240\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") "
Oct 31 04:09:31.340566 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.340425    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-maas-api-tls\") pod \"a07d7017-aef8-49fb-8de0-0b3cba42c240\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") "
Oct 31 04:09:31.340566 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.340461    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-metrics-certs\") pod \"a07d7017-aef8-49fb-8de0-0b3cba42c240\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") "
Oct 31 04:09:31.340566 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.340555    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a07d7017-aef8-49fb-8de0-0b3cba42c240-kube-api-access-lmwg6\" (UniqueName: \"kubernetes.io/projected/a07d7017-aef8-49fb-8de0-0b3cba42c240-kube-api-access-lmwg6\") pod \"a07d7017-aef8-49fb-8de0-0b3cba42c240\" (UID: \"a07d7017-aef8-49fb-8de0-0b3cba42c240\") "
Oct 31 04:09:31.340811 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.340781    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a07d7017-aef8-49fb-8de0-0b3cba42c240-openshift-service-ca" pod "a07d7017-aef8-49fb-8de0-0b3cba42c240" (UID: "a07d7017-aef8-49fb-8de0-0b3cba42c240"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:09:31.342777 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.342743    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-metrics-certs" pod "a07d7017-aef8-49fb-8de0-0b3cba42c240" (UID: "a07d7017-aef8-49fb-8de0-0b3cba42c240"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:09:31.342777 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.342761    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-maas-api-tls" pod "a07d7017-aef8-49fb-8de0-0b3cba42c240" (UID: "a07d7017-aef8-49fb-8de0-0b3cba42c240"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:09:31.342935 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.342841    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a07d7017-aef8-49fb-8de0-0b3cba42c240-kube-api-access-lmwg6" pod "a07d7017-aef8-49fb-8de0-0b3cba42c240" (UID: "a07d7017-aef8-49fb-8de0-0b3cba42c240"). InnerVolumeSpecName "kube-api-access-lmwg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:09:31.441750 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.441715    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmwg6\" (UniqueName: \"kubernetes.io/projected/a07d7017-aef8-49fb-8de0-0b3cba42c240-kube-api-access-lmwg6\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:31.441750 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.441749    4926 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a07d7017-aef8-49fb-8de0-0b3cba42c240-openshift-service-ca\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:31.441949 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.441765    4926 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-maas-api-tls\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:31.441949 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.441780    4926 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a07d7017-aef8-49fb-8de0-0b3cba42c240-metrics-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:31.443775 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.443746    4926 generic.go:356] "Generic (PLEG): container finished" podID="a07d7017-aef8-49fb-8de0-0b3cba42c240" containerID="8ef6ee32cff2ffa1f047256fc76443df5b460280cefc7eb07207d45b8fe32aa2" exitCode=0
Oct 31 04:09:31.443897 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.443785    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv" event={"ID":"a07d7017-aef8-49fb-8de0-0b3cba42c240","Type":"ContainerDied","Data":"8ef6ee32cff2ffa1f047256fc76443df5b460280cefc7eb07207d45b8fe32aa2"}
Oct 31 04:09:31.443897 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.443802    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"
Oct 31 04:09:31.443897 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.443834    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv" event={"ID":"a07d7017-aef8-49fb-8de0-0b3cba42c240","Type":"ContainerDied","Data":"3feb40cc56585804a33198ee0a2eb642112383820af4660baa56c52331af193e"}
Oct 31 04:09:31.443897 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.443864    4926 scope.go:122] "RemoveContainer" containerID="8ef6ee32cff2ffa1f047256fc76443df5b460280cefc7eb07207d45b8fe32aa2"
Oct 31 04:09:31.453527 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.453507    4926 scope.go:122] "RemoveContainer" containerID="8ef6ee32cff2ffa1f047256fc76443df5b460280cefc7eb07207d45b8fe32aa2"
Oct 31 04:09:31.453786 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:09:31.453768    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ef6ee32cff2ffa1f047256fc76443df5b460280cefc7eb07207d45b8fe32aa2\": container with ID starting with 8ef6ee32cff2ffa1f047256fc76443df5b460280cefc7eb07207d45b8fe32aa2 not found: ID does not exist" containerID="8ef6ee32cff2ffa1f047256fc76443df5b460280cefc7eb07207d45b8fe32aa2"
Oct 31 04:09:31.453847 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.453798    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ef6ee32cff2ffa1f047256fc76443df5b460280cefc7eb07207d45b8fe32aa2"} err="failed to get container status \"8ef6ee32cff2ffa1f047256fc76443df5b460280cefc7eb07207d45b8fe32aa2\": rpc error: code = NotFound desc = could not find container \"8ef6ee32cff2ffa1f047256fc76443df5b460280cefc7eb07207d45b8fe32aa2\": container with ID starting with 8ef6ee32cff2ffa1f047256fc76443df5b460280cefc7eb07207d45b8fe32aa2 not found: ID does not exist"
Oct 31 04:09:31.614885 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.614796    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"]
Oct 31 04:09:31.617138 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:31.617109    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-3473a4-645f64f8c4-sqtwv"]
Oct 31 04:09:33.537765 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:33.537672    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a07d7017-aef8-49fb-8de0-0b3cba42c240" path="/var/lib/kubelet/pods/a07d7017-aef8-49fb-8de0-0b3cba42c240/volumes"
Oct 31 04:09:35.313072 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:09:35.313030    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:09:45.833711 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:45.833667    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"]
Oct 31 04:09:45.834222 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:45.833931    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp" podUID="c9f51242-834d-4824-b9c9-05315f436249" containerName="maas-api" containerID="cri-o://cb1352404c0ff772720c1f7bfcb732792da74a2532b456725dc2685dcb63b7c4" gracePeriod=30
Oct 31 04:09:46.125416 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.125390    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:09:46.195291 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.192288    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c9f51242-834d-4824-b9c9-05315f436249-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9f51242-834d-4824-b9c9-05315f436249-metrics-certs\") pod \"c9f51242-834d-4824-b9c9-05315f436249\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") "
Oct 31 04:09:46.195291 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.192940    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9f51242-834d-4824-b9c9-05315f436249-openshift-service-ca" pod "c9f51242-834d-4824-b9c9-05315f436249" (UID: "c9f51242-834d-4824-b9c9-05315f436249"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:09:46.195291 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.193000    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9f51242-834d-4824-b9c9-05315f436249-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9f51242-834d-4824-b9c9-05315f436249-openshift-service-ca\") pod \"c9f51242-834d-4824-b9c9-05315f436249\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") "
Oct 31 04:09:46.195291 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.193074    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9f51242-834d-4824-b9c9-05315f436249-kube-api-access-5dpk7\" (UniqueName: \"kubernetes.io/projected/c9f51242-834d-4824-b9c9-05315f436249-kube-api-access-5dpk7\") pod \"c9f51242-834d-4824-b9c9-05315f436249\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") "
Oct 31 04:09:46.195291 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.193154    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c9f51242-834d-4824-b9c9-05315f436249-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9f51242-834d-4824-b9c9-05315f436249-maas-api-tls\") pod \"c9f51242-834d-4824-b9c9-05315f436249\" (UID: \"c9f51242-834d-4824-b9c9-05315f436249\") "
Oct 31 04:09:46.195291 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.193645    4926 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9f51242-834d-4824-b9c9-05315f436249-openshift-service-ca\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:46.195291 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.195145    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9f51242-834d-4824-b9c9-05315f436249-metrics-certs" pod "c9f51242-834d-4824-b9c9-05315f436249" (UID: "c9f51242-834d-4824-b9c9-05315f436249"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:09:46.199286 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.195928    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9f51242-834d-4824-b9c9-05315f436249-kube-api-access-5dpk7" pod "c9f51242-834d-4824-b9c9-05315f436249" (UID: "c9f51242-834d-4824-b9c9-05315f436249"). InnerVolumeSpecName "kube-api-access-5dpk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:09:46.199286 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.195982    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9f51242-834d-4824-b9c9-05315f436249-maas-api-tls" pod "c9f51242-834d-4824-b9c9-05315f436249" (UID: "c9f51242-834d-4824-b9c9-05315f436249"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:09:46.295051 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.295008    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dpk7\" (UniqueName: \"kubernetes.io/projected/c9f51242-834d-4824-b9c9-05315f436249-kube-api-access-5dpk7\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:46.295051 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.295044    4926 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9f51242-834d-4824-b9c9-05315f436249-maas-api-tls\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:46.295051 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.295057    4926 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9f51242-834d-4824-b9c9-05315f436249-metrics-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:09:46.528213 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.528172    4926 generic.go:356] "Generic (PLEG): container finished" podID="c9f51242-834d-4824-b9c9-05315f436249" containerID="cb1352404c0ff772720c1f7bfcb732792da74a2532b456725dc2685dcb63b7c4" exitCode=0
Oct 31 04:09:46.528213 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.528224    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"
Oct 31 04:09:46.528465 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.528235    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp" event={"ID":"c9f51242-834d-4824-b9c9-05315f436249","Type":"ContainerDied","Data":"cb1352404c0ff772720c1f7bfcb732792da74a2532b456725dc2685dcb63b7c4"}
Oct 31 04:09:46.528465 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.528307    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp" event={"ID":"c9f51242-834d-4824-b9c9-05315f436249","Type":"ContainerDied","Data":"f45bdb61fb5409ffab5ac58b383494258bed537979f5ec8df8adcd88ed9f38be"}
Oct 31 04:09:46.528465 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.528338    4926 scope.go:122] "RemoveContainer" containerID="cb1352404c0ff772720c1f7bfcb732792da74a2532b456725dc2685dcb63b7c4"
Oct 31 04:09:46.543659 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.543637    4926 scope.go:122] "RemoveContainer" containerID="cb1352404c0ff772720c1f7bfcb732792da74a2532b456725dc2685dcb63b7c4"
Oct 31 04:09:46.543932 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:09:46.543904    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb1352404c0ff772720c1f7bfcb732792da74a2532b456725dc2685dcb63b7c4\": container with ID starting with cb1352404c0ff772720c1f7bfcb732792da74a2532b456725dc2685dcb63b7c4 not found: ID does not exist" containerID="cb1352404c0ff772720c1f7bfcb732792da74a2532b456725dc2685dcb63b7c4"
Oct 31 04:09:46.544036 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.543938    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb1352404c0ff772720c1f7bfcb732792da74a2532b456725dc2685dcb63b7c4"} err="failed to get container status \"cb1352404c0ff772720c1f7bfcb732792da74a2532b456725dc2685dcb63b7c4\": rpc error: code = NotFound desc = could not find container \"cb1352404c0ff772720c1f7bfcb732792da74a2532b456725dc2685dcb63b7c4\": container with ID starting with cb1352404c0ff772720c1f7bfcb732792da74a2532b456725dc2685dcb63b7c4 not found: ID does not exist"
Oct 31 04:09:46.707834 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.707796    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"]
Oct 31 04:09:46.711343 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:46.711311    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-288b63-76cc5d597d-m7hzp"]
Oct 31 04:09:47.538801 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:09:47.538763    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9f51242-834d-4824-b9c9-05315f436249" path="/var/lib/kubelet/pods/c9f51242-834d-4824-b9c9-05315f436249/volumes"
Oct 31 04:10:00.320968 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:00.320931    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"]
Oct 31 04:10:00.373595 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:00.373556    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"]
Oct 31 04:10:00.376328 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:00.376297    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"]
Oct 31 04:10:05.313513 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:10:05.313470    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:10:15.341243 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.341170    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerName="main" containerID="cri-o://d7bae8971d3f73a64c66de43ed6039e25719a477b0bd614f7f932d706b8dd097" gracePeriod=45
Oct 31 04:10:15.394411 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.394342    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" podUID="ade2ace8-0217-404c-9592-85576f16b085" containerName="main" containerID="cri-o://0889eadb61cbb22dce35033ee7e8bf29945c2475191c2d928a954d9054733969" gracePeriod=45
Oct 31 04:10:15.399671 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.399634    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" podUID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" containerName="main" containerID="cri-o://1d30889e6ccc2fcac9dbbb123f2d4f6806aa14d3c043f951ae1c146a0d974185" gracePeriod=45
Oct 31 04:10:15.682422 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.682232    4926 generic.go:356] "Generic (PLEG): container finished" podID="ade2ace8-0217-404c-9592-85576f16b085" containerID="0889eadb61cbb22dce35033ee7e8bf29945c2475191c2d928a954d9054733969" exitCode=0
Oct 31 04:10:15.682422 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.682356    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" event={"ID":"ade2ace8-0217-404c-9592-85576f16b085","Type":"ContainerDied","Data":"0889eadb61cbb22dce35033ee7e8bf29945c2475191c2d928a954d9054733969"}
Oct 31 04:10:15.682422 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.682394    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm" event={"ID":"ade2ace8-0217-404c-9592-85576f16b085","Type":"ContainerDied","Data":"40d498f1d2049b2a22e7ac92e03fa0220fb319ca15ae39ebe19a464fc17d3fb9"}
Oct 31 04:10:15.682422 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.682410    4926 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40d498f1d2049b2a22e7ac92e03fa0220fb319ca15ae39ebe19a464fc17d3fb9"
Oct 31 04:10:15.684191 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.684164    4926 generic.go:356] "Generic (PLEG): container finished" podID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerID="d7bae8971d3f73a64c66de43ed6039e25719a477b0bd614f7f932d706b8dd097" exitCode=0
Oct 31 04:10:15.684343 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.684230    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" event={"ID":"9f0348bd-b29e-49eb-802a-dfcb95413b8a","Type":"ContainerDied","Data":"d7bae8971d3f73a64c66de43ed6039e25719a477b0bd614f7f932d706b8dd097"}
Oct 31 04:10:15.684343 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.684294    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd" event={"ID":"9f0348bd-b29e-49eb-802a-dfcb95413b8a","Type":"ContainerDied","Data":"fa879cd910817c5b380a1cdb9aa0618de4404e3cd4383f534fdfc28e39924580"}
Oct 31 04:10:15.684343 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.684312    4926 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fa879cd910817c5b380a1cdb9aa0618de4404e3cd4383f534fdfc28e39924580"
Oct 31 04:10:15.686528 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.686509    4926 generic.go:356] "Generic (PLEG): container finished" podID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" containerID="1d30889e6ccc2fcac9dbbb123f2d4f6806aa14d3c043f951ae1c146a0d974185" exitCode=0
Oct 31 04:10:15.686792 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.686574    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" event={"ID":"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228","Type":"ContainerDied","Data":"1d30889e6ccc2fcac9dbbb123f2d4f6806aa14d3c043f951ae1c146a0d974185"}
Oct 31 04:10:15.693677 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.693654    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 04:10:15.697252 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.697232    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 04:10:15.717448 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.717428    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ade2ace8-0217-404c-9592-85576f16b085-kube-api-access-789dv\" (UniqueName: \"kubernetes.io/projected/ade2ace8-0217-404c-9592-85576f16b085-kube-api-access-789dv\") pod \"ade2ace8-0217-404c-9592-85576f16b085\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") "
Oct 31 04:10:15.717562 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.717496    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ade2ace8-0217-404c-9592-85576f16b085-tls-certs\" (UniqueName: \"kubernetes.io/secret/ade2ace8-0217-404c-9592-85576f16b085-tls-certs\") pod \"ade2ace8-0217-404c-9592-85576f16b085\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") "
Oct 31 04:10:15.717639 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.717572    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f0348bd-b29e-49eb-802a-dfcb95413b8a-kube-api-access-w2bkd\" (UniqueName: \"kubernetes.io/projected/9f0348bd-b29e-49eb-802a-dfcb95413b8a-kube-api-access-w2bkd\") pod \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") "
Oct 31 04:10:15.717639 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.717609    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-dshm\") pod \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") "
Oct 31 04:10:15.717745 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.717646    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-model-cache\") pod \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") "
Oct 31 04:10:15.717745 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.717686    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-tmp-dir\") pod \"ade2ace8-0217-404c-9592-85576f16b085\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") "
Oct 31 04:10:15.717745 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.717727    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-model-cache\") pod \"ade2ace8-0217-404c-9592-85576f16b085\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") "
Oct 31 04:10:15.717894 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.717766    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-tmp-dir\") pod \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") "
Oct 31 04:10:15.717894 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.717864    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-home\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-home\") pod \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") "
Oct 31 04:10:15.718015 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.717911    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-home\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-home\") pod \"ade2ace8-0217-404c-9592-85576f16b085\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") "
Oct 31 04:10:15.718015 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.717944    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f0348bd-b29e-49eb-802a-dfcb95413b8a-tls-certs\" (UniqueName: \"kubernetes.io/secret/9f0348bd-b29e-49eb-802a-dfcb95413b8a-tls-certs\") pod \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\" (UID: \"9f0348bd-b29e-49eb-802a-dfcb95413b8a\") "
Oct 31 04:10:15.718015 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.717980    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-dshm\") pod \"ade2ace8-0217-404c-9592-85576f16b085\" (UID: \"ade2ace8-0217-404c-9592-85576f16b085\") "
Oct 31 04:10:15.719108 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.719079    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-tmp-dir" pod "ade2ace8-0217-404c-9592-85576f16b085" (UID: "ade2ace8-0217-404c-9592-85576f16b085"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.719298 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.719251    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-model-cache" pod "ade2ace8-0217-404c-9592-85576f16b085" (UID: "ade2ace8-0217-404c-9592-85576f16b085"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.719444 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.719427    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-home" pod "ade2ace8-0217-404c-9592-85576f16b085" (UID: "ade2ace8-0217-404c-9592-85576f16b085"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.719590 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.719575    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-home" pod "9f0348bd-b29e-49eb-802a-dfcb95413b8a" (UID: "9f0348bd-b29e-49eb-802a-dfcb95413b8a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.720329 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.720234    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-model-cache" pod "9f0348bd-b29e-49eb-802a-dfcb95413b8a" (UID: "9f0348bd-b29e-49eb-802a-dfcb95413b8a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.720433 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.720391    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ade2ace8-0217-404c-9592-85576f16b085-kube-api-access-789dv" pod "ade2ace8-0217-404c-9592-85576f16b085" (UID: "ade2ace8-0217-404c-9592-85576f16b085"). InnerVolumeSpecName "kube-api-access-789dv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:10:15.720573 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.720538    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-tmp-dir" pod "9f0348bd-b29e-49eb-802a-dfcb95413b8a" (UID: "9f0348bd-b29e-49eb-802a-dfcb95413b8a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.720836 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.720745    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-dshm" pod "9f0348bd-b29e-49eb-802a-dfcb95413b8a" (UID: "9f0348bd-b29e-49eb-802a-dfcb95413b8a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.720836 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.720803    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-dshm" pod "ade2ace8-0217-404c-9592-85576f16b085" (UID: "ade2ace8-0217-404c-9592-85576f16b085"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.722120 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.722099    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f0348bd-b29e-49eb-802a-dfcb95413b8a-tls-certs" pod "9f0348bd-b29e-49eb-802a-dfcb95413b8a" (UID: "9f0348bd-b29e-49eb-802a-dfcb95413b8a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:10:15.722213 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.722164    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ade2ace8-0217-404c-9592-85576f16b085-tls-certs" pod "ade2ace8-0217-404c-9592-85576f16b085" (UID: "ade2ace8-0217-404c-9592-85576f16b085"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:10:15.726319 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.726297    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f0348bd-b29e-49eb-802a-dfcb95413b8a-kube-api-access-w2bkd" pod "9f0348bd-b29e-49eb-802a-dfcb95413b8a" (UID: "9f0348bd-b29e-49eb-802a-dfcb95413b8a"). InnerVolumeSpecName "kube-api-access-w2bkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:10:15.734662 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.734644    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 04:10:15.819340 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.819308    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-dshm\") pod \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") "
Oct 31 04:10:15.819558 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.819395    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-model-cache\") pod \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") "
Oct 31 04:10:15.819558 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.819459    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-home\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-home\") pod \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") "
Oct 31 04:10:15.819558 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.819504    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-tmp-dir\") pod \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") "
Oct 31 04:10:15.819738 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.819577    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-kube-api-access-4bv2k\" (UniqueName: \"kubernetes.io/projected/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-kube-api-access-4bv2k\") pod \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") "
Oct 31 04:10:15.819738 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.819613    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-tls-certs\" (UniqueName: \"kubernetes.io/secret/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-tls-certs\") pod \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\" (UID: \"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228\") "
Oct 31 04:10:15.819886 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.819847    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-tmp-dir" pod "a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" (UID: "a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.819943 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.819918    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-model-cache" pod "a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" (UID: "a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.820330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.820082    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.820330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.820108    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.820330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.820181    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.820330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.820194    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.820330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.820208    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.820330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.820221    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.820330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.820234    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.820330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.820231    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-home" pod "a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" (UID: "a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.820330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.820246    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.820330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.820260    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f0348bd-b29e-49eb-802a-dfcb95413b8a-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.820330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.820290    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ade2ace8-0217-404c-9592-85576f16b085-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.820330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.820304    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-789dv\" (UniqueName: \"kubernetes.io/projected/ade2ace8-0217-404c-9592-85576f16b085-kube-api-access-789dv\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.820330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.820318    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ade2ace8-0217-404c-9592-85576f16b085-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.820330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.820333    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2bkd\" (UniqueName: \"kubernetes.io/projected/9f0348bd-b29e-49eb-802a-dfcb95413b8a-kube-api-access-w2bkd\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.821336 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.820346    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f0348bd-b29e-49eb-802a-dfcb95413b8a-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.821748 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.821724    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-dshm" pod "a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" (UID: "a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:10:15.822676 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.822655    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-kube-api-access-4bv2k" pod "a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" (UID: "a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228"). InnerVolumeSpecName "kube-api-access-4bv2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:10:15.822861 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.822687    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-tls-certs" pod "a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" (UID: "a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:10:15.921395 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.921324    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bv2k\" (UniqueName: \"kubernetes.io/projected/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-kube-api-access-4bv2k\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.921395 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.921351    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.921395 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.921360    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:15.921395 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:15.921368    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:16.693238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:16.693153    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"
Oct 31 04:10:16.693238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:16.693169    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"
Oct 31 04:10:16.693238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:16.693203    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8" event={"ID":"a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228","Type":"ContainerDied","Data":"d5eca5e57079588fab30da2a46ea0d8411f548982f70396a8e316d3c7150b924"}
Oct 31 04:10:16.693834 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:16.693244    4926 scope.go:122] "RemoveContainer" containerID="1d30889e6ccc2fcac9dbbb123f2d4f6806aa14d3c043f951ae1c146a0d974185"
Oct 31 04:10:16.693834 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:16.693300    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"
Oct 31 04:10:16.861737 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:16.861705    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"]
Oct 31 04:10:16.912980 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:16.912948    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/distinct-w0-ae8e27-kserve-74bf68f68c-mbhkd"]
Oct 31 04:10:16.977388 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:16.977357    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"]
Oct 31 04:10:16.981201 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:16.981174    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/simulator-w0-ae8e27-kserve-56cbd4d949-nfgl8"]
Oct 31 04:10:17.095350 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:17.095317    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"]
Oct 31 04:10:17.099490 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:17.099466    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-ae8e27/premium-w0-ae8e27-kserve-75bbdb9c68-f4wjm"]
Oct 31 04:10:17.537575 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:17.537547    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" path="/var/lib/kubelet/pods/9f0348bd-b29e-49eb-802a-dfcb95413b8a/volumes"
Oct 31 04:10:17.537981 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:17.537966    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" path="/var/lib/kubelet/pods/a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228/volumes"
Oct 31 04:10:17.538388 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:17.538375    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ade2ace8-0217-404c-9592-85576f16b085" path="/var/lib/kubelet/pods/ade2ace8-0217-404c-9592-85576f16b085/volumes"
Oct 31 04:10:24.121469 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:24.121428    4926 scope.go:122] "RemoveContainer" containerID="0889eadb61cbb22dce35033ee7e8bf29945c2475191c2d928a954d9054733969"
Oct 31 04:10:24.130242 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:24.130221    4926 scope.go:122] "RemoveContainer" containerID="d7bae8971d3f73a64c66de43ed6039e25719a477b0bd614f7f932d706b8dd097"
Oct 31 04:10:35.313645 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:10:35.313604    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:10:50.253888 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.253847    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"]
Oct 31 04:10:50.256345 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.254075    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4" podUID="74e17325-bf76-4639-9475-b54f780ed429" containerName="maas-api" containerID="cri-o://7c8dc3220d4c979dfd23aa746297408d4c36935963d4c0aa6c3cfa7a9d3b73a7" gracePeriod=30
Oct 31 04:10:50.519915 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.519852    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 04:10:50.574038 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.574007    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74e17325-bf76-4639-9475-b54f780ed429-kube-api-access-kpsv8\" (UniqueName: \"kubernetes.io/projected/74e17325-bf76-4639-9475-b54f780ed429-kube-api-access-kpsv8\") pod \"74e17325-bf76-4639-9475-b54f780ed429\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") "
Oct 31 04:10:50.574242 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.574067    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74e17325-bf76-4639-9475-b54f780ed429-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74e17325-bf76-4639-9475-b54f780ed429-openshift-service-ca\") pod \"74e17325-bf76-4639-9475-b54f780ed429\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") "
Oct 31 04:10:50.574242 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.574093    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-metrics-certs\" (UniqueName: \"kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-metrics-certs\") pod \"74e17325-bf76-4639-9475-b54f780ed429\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") "
Oct 31 04:10:50.574242 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.574134    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-maas-api-tls\") pod \"74e17325-bf76-4639-9475-b54f780ed429\" (UID: \"74e17325-bf76-4639-9475-b54f780ed429\") "
Oct 31 04:10:50.574563 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.574538    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74e17325-bf76-4639-9475-b54f780ed429-openshift-service-ca" pod "74e17325-bf76-4639-9475-b54f780ed429" (UID: "74e17325-bf76-4639-9475-b54f780ed429"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:10:50.576510 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.576480    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-metrics-certs" pod "74e17325-bf76-4639-9475-b54f780ed429" (UID: "74e17325-bf76-4639-9475-b54f780ed429"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:10:50.576510 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.576489    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74e17325-bf76-4639-9475-b54f780ed429-kube-api-access-kpsv8" pod "74e17325-bf76-4639-9475-b54f780ed429" (UID: "74e17325-bf76-4639-9475-b54f780ed429"). InnerVolumeSpecName "kube-api-access-kpsv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:10:50.576644 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.576507    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-maas-api-tls" pod "74e17325-bf76-4639-9475-b54f780ed429" (UID: "74e17325-bf76-4639-9475-b54f780ed429"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:10:50.675312 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.675256    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpsv8\" (UniqueName: \"kubernetes.io/projected/74e17325-bf76-4639-9475-b54f780ed429-kube-api-access-kpsv8\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:50.675312 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.675310    4926 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/74e17325-bf76-4639-9475-b54f780ed429-openshift-service-ca\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:50.675312 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.675321    4926 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-metrics-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:50.675544 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.675330    4926 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/74e17325-bf76-4639-9475-b54f780ed429-maas-api-tls\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:10:50.875579 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.875491    4926 generic.go:356] "Generic (PLEG): container finished" podID="74e17325-bf76-4639-9475-b54f780ed429" containerID="7c8dc3220d4c979dfd23aa746297408d4c36935963d4c0aa6c3cfa7a9d3b73a7" exitCode=0
Oct 31 04:10:50.875579 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.875543    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"
Oct 31 04:10:50.875801 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.875575    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4" event={"ID":"74e17325-bf76-4639-9475-b54f780ed429","Type":"ContainerDied","Data":"7c8dc3220d4c979dfd23aa746297408d4c36935963d4c0aa6c3cfa7a9d3b73a7"}
Oct 31 04:10:50.875801 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.875628    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4" event={"ID":"74e17325-bf76-4639-9475-b54f780ed429","Type":"ContainerDied","Data":"7982cb81b9ac074162ace0f226fa2497632189db3f519da046746012d57f734e"}
Oct 31 04:10:50.875801 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.875649    4926 scope.go:122] "RemoveContainer" containerID="7c8dc3220d4c979dfd23aa746297408d4c36935963d4c0aa6c3cfa7a9d3b73a7"
Oct 31 04:10:50.885495 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.885477    4926 scope.go:122] "RemoveContainer" containerID="7c8dc3220d4c979dfd23aa746297408d4c36935963d4c0aa6c3cfa7a9d3b73a7"
Oct 31 04:10:50.885822 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:10:50.885800    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c8dc3220d4c979dfd23aa746297408d4c36935963d4c0aa6c3cfa7a9d3b73a7\": container with ID starting with 7c8dc3220d4c979dfd23aa746297408d4c36935963d4c0aa6c3cfa7a9d3b73a7 not found: ID does not exist" containerID="7c8dc3220d4c979dfd23aa746297408d4c36935963d4c0aa6c3cfa7a9d3b73a7"
Oct 31 04:10:50.885883 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:50.885833    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c8dc3220d4c979dfd23aa746297408d4c36935963d4c0aa6c3cfa7a9d3b73a7"} err="failed to get container status \"7c8dc3220d4c979dfd23aa746297408d4c36935963d4c0aa6c3cfa7a9d3b73a7\": rpc error: code = NotFound desc = could not find container \"7c8dc3220d4c979dfd23aa746297408d4c36935963d4c0aa6c3cfa7a9d3b73a7\": container with ID starting with 7c8dc3220d4c979dfd23aa746297408d4c36935963d4c0aa6c3cfa7a9d3b73a7 not found: ID does not exist"
Oct 31 04:10:51.044769 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:51.044737    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"]
Oct 31 04:10:51.048632 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:51.048603    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-ae8e27-659f45d6ff-r94s4"]
Oct 31 04:10:51.543323 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:51.543291    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74e17325-bf76-4639-9475-b54f780ed429" path="/var/lib/kubelet/pods/74e17325-bf76-4639-9475-b54f780ed429/volumes"
Oct 31 04:10:59.366155 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:59.366122    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"]
Oct 31 04:10:59.366570 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:10:59.366391    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7" podUID="ff3deeee-d307-4df0-8656-dcc3a13e060a" containerName="istio-proxy" containerID="cri-o://7fe66428b9f5afbec05882f58f08d66ea5f1c0dda207bde5fe4e4273183993c0" gracePeriod=30
Oct 31 04:11:01.529963 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:01.529930    4926 patch_prober.go:29] interesting pod/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:11:01.530369 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:01.529990    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7" podUID="ff3deeee-d307-4df0-8656-dcc3a13e060a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:11:04.622859 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.622834    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 04:11:04.718577 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.718545    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff3deeee-d307-4df0-8656-dcc3a13e060a-kube-api-access-thb9h\" (UniqueName: \"kubernetes.io/projected/ff3deeee-d307-4df0-8656-dcc3a13e060a-kube-api-access-thb9h\") pod \"ff3deeee-d307-4df0-8656-dcc3a13e060a\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") "
Oct 31 04:11:04.718761 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.718606    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-envoy\") pod \"ff3deeee-d307-4df0-8656-dcc3a13e060a\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") "
Oct 31 04:11:04.718761 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.718641    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-credential-socket\") pod \"ff3deeee-d307-4df0-8656-dcc3a13e060a\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") "
Oct 31 04:11:04.718761 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.718740    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-data\") pod \"ff3deeee-d307-4df0-8656-dcc3a13e060a\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") "
Oct 31 04:11:04.718940 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.718786    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-podinfo\") pod \"ff3deeee-d307-4df0-8656-dcc3a13e060a\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") "
Oct 31 04:11:04.718940 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.718835    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-token\" (UniqueName: \"kubernetes.io/projected/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-token\") pod \"ff3deeee-d307-4df0-8656-dcc3a13e060a\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") "
Oct 31 04:11:04.718940 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.718886    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-workload-certs\") pod \"ff3deeee-d307-4df0-8656-dcc3a13e060a\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") "
Oct 31 04:11:04.719102 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.718958    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff3deeee-d307-4df0-8656-dcc3a13e060a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff3deeee-d307-4df0-8656-dcc3a13e060a-istiod-ca-cert\") pod \"ff3deeee-d307-4df0-8656-dcc3a13e060a\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") "
Oct 31 04:11:04.719102 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.719003    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-workload-socket\") pod \"ff3deeee-d307-4df0-8656-dcc3a13e060a\" (UID: \"ff3deeee-d307-4df0-8656-dcc3a13e060a\") "
Oct 31 04:11:04.719102 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.719027    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-data" pod "ff3deeee-d307-4df0-8656-dcc3a13e060a" (UID: "ff3deeee-d307-4df0-8656-dcc3a13e060a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:04.719102 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.719098    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-workload-certs" pod "ff3deeee-d307-4df0-8656-dcc3a13e060a" (UID: "ff3deeee-d307-4df0-8656-dcc3a13e060a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:04.719334 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.719170    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-credential-socket" pod "ff3deeee-d307-4df0-8656-dcc3a13e060a" (UID: "ff3deeee-d307-4df0-8656-dcc3a13e060a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:04.719549 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.719389    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-workload-socket" pod "ff3deeee-d307-4df0-8656-dcc3a13e060a" (UID: "ff3deeee-d307-4df0-8656-dcc3a13e060a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:04.719549 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.719427    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff3deeee-d307-4df0-8656-dcc3a13e060a-istiod-ca-cert" pod "ff3deeee-d307-4df0-8656-dcc3a13e060a" (UID: "ff3deeee-d307-4df0-8656-dcc3a13e060a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:11:04.719549 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.719470    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.719549 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.719495    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.719549 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.719511    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.719549 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.719525    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.721148 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.721122    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff3deeee-d307-4df0-8656-dcc3a13e060a-kube-api-access-thb9h" pod "ff3deeee-d307-4df0-8656-dcc3a13e060a" (UID: "ff3deeee-d307-4df0-8656-dcc3a13e060a"). InnerVolumeSpecName "kube-api-access-thb9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:04.721310 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.721186    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-podinfo" pod "ff3deeee-d307-4df0-8656-dcc3a13e060a" (UID: "ff3deeee-d307-4df0-8656-dcc3a13e060a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:11:04.721310 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.721255    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-token" pod "ff3deeee-d307-4df0-8656-dcc3a13e060a" (UID: "ff3deeee-d307-4df0-8656-dcc3a13e060a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:04.721428 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.721336    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-envoy" pod "ff3deeee-d307-4df0-8656-dcc3a13e060a" (UID: "ff3deeee-d307-4df0-8656-dcc3a13e060a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:04.820039 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.819993    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thb9h\" (UniqueName: \"kubernetes.io/projected/ff3deeee-d307-4df0-8656-dcc3a13e060a-kube-api-access-thb9h\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.820039 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.820025    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.820039 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.820035    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.820039 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.820044    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff3deeee-d307-4df0-8656-dcc3a13e060a-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.820481 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.820054    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff3deeee-d307-4df0-8656-dcc3a13e060a-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:04.944231 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.944198    4926 generic.go:356] "Generic (PLEG): container finished" podID="ff3deeee-d307-4df0-8656-dcc3a13e060a" containerID="7fe66428b9f5afbec05882f58f08d66ea5f1c0dda207bde5fe4e4273183993c0" exitCode=0
Oct 31 04:11:04.944441 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.944246    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"
Oct 31 04:11:04.944441 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.944306    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7" event={"ID":"ff3deeee-d307-4df0-8656-dcc3a13e060a","Type":"ContainerDied","Data":"7fe66428b9f5afbec05882f58f08d66ea5f1c0dda207bde5fe4e4273183993c0"}
Oct 31 04:11:04.944441 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.944367    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7" event={"ID":"ff3deeee-d307-4df0-8656-dcc3a13e060a","Type":"ContainerDied","Data":"7f0595c5198fd5df6c1b928776a3f61bb29c6fa3d34ea4169c35a67dd354a6d0"}
Oct 31 04:11:04.944441 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.944389    4926 scope.go:122] "RemoveContainer" containerID="7fe66428b9f5afbec05882f58f08d66ea5f1c0dda207bde5fe4e4273183993c0"
Oct 31 04:11:04.954455 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.954436    4926 scope.go:122] "RemoveContainer" containerID="7fe66428b9f5afbec05882f58f08d66ea5f1c0dda207bde5fe4e4273183993c0"
Oct 31 04:11:04.954797 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:11:04.954778    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fe66428b9f5afbec05882f58f08d66ea5f1c0dda207bde5fe4e4273183993c0\": container with ID starting with 7fe66428b9f5afbec05882f58f08d66ea5f1c0dda207bde5fe4e4273183993c0 not found: ID does not exist" containerID="7fe66428b9f5afbec05882f58f08d66ea5f1c0dda207bde5fe4e4273183993c0"
Oct 31 04:11:04.954848 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:04.954807    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fe66428b9f5afbec05882f58f08d66ea5f1c0dda207bde5fe4e4273183993c0"} err="failed to get container status \"7fe66428b9f5afbec05882f58f08d66ea5f1c0dda207bde5fe4e4273183993c0\": rpc error: code = NotFound desc = could not find container \"7fe66428b9f5afbec05882f58f08d66ea5f1c0dda207bde5fe4e4273183993c0\": container with ID starting with 7fe66428b9f5afbec05882f58f08d66ea5f1c0dda207bde5fe4e4273183993c0 not found: ID does not exist"
Oct 31 04:11:05.114306 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:05.114207    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"]
Oct 31 04:11:05.120675 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:05.120644    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ae8e27-openshift-default-c87fdc98f-fnfg7"]
Oct 31 04:11:05.313003 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:11:05.312961    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:11:05.537810 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:05.537775    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff3deeee-d307-4df0-8656-dcc3a13e060a" path="/var/lib/kubelet/pods/ff3deeee-d307-4df0-8656-dcc3a13e060a/volumes"
Oct 31 04:11:12.189574 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.189537    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-6968-07b6b7"]
Oct 31 04:11:12.189972 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.189920    4926 cpu_manager.go:414] "containerMap: removing container" podUID="7ce8937f-8f05-4e63-85bb-5ee282b5367b" containerName="istio-proxy"
Oct 31 04:11:12.189972 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.189933    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ce8937f-8f05-4e63-85bb-5ee282b5367b" containerName="istio-proxy"
Oct 31 04:11:12.189972 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.189945    4926 cpu_manager.go:414] "containerMap: removing container" podUID="a07d7017-aef8-49fb-8de0-0b3cba42c240" containerName="maas-api"
Oct 31 04:11:12.189972 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.189950    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a07d7017-aef8-49fb-8de0-0b3cba42c240" containerName="maas-api"
Oct 31 04:11:12.189972 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.189961    4926 cpu_manager.go:414] "containerMap: removing container" podUID="a65ad9de-668e-4600-8643-546a6ce94b99" containerName="istio-proxy"
Oct 31 04:11:12.189972 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.189966    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a65ad9de-668e-4600-8643-546a6ce94b99" containerName="istio-proxy"
Oct 31 04:11:12.189972 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.189976    4926 cpu_manager.go:414] "containerMap: removing container" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerName="main"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.189981    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerName="main"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.189987    4926 cpu_manager.go:414] "containerMap: removing container" podUID="a9f9a022-fa1c-4f08-ac54-cb751f629547" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.189991    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9f9a022-fa1c-4f08-ac54-cb751f629547" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190005    4926 cpu_manager.go:414] "containerMap: removing container" podUID="c9f51242-834d-4824-b9c9-05315f436249" containerName="maas-api"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190009    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9f51242-834d-4824-b9c9-05315f436249" containerName="maas-api"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190021    4926 cpu_manager.go:414] "containerMap: removing container" podUID="ade2ace8-0217-404c-9592-85576f16b085" containerName="main"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190025    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ade2ace8-0217-404c-9592-85576f16b085" containerName="main"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190031    4926 cpu_manager.go:414] "containerMap: removing container" podUID="0664091e-7353-433c-bd14-76c8b1b6d795" containerName="maas-api"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190035    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0664091e-7353-433c-bd14-76c8b1b6d795" containerName="maas-api"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190045    4926 cpu_manager.go:414] "containerMap: removing container" podUID="7ce8937f-8f05-4e63-85bb-5ee282b5367b" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190050    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ce8937f-8f05-4e63-85bb-5ee282b5367b" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190057    4926 cpu_manager.go:414] "containerMap: removing container" podUID="ff3deeee-d307-4df0-8656-dcc3a13e060a" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190061    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff3deeee-d307-4df0-8656-dcc3a13e060a" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190070    4926 cpu_manager.go:414] "containerMap: removing container" podUID="74e17325-bf76-4639-9475-b54f780ed429" containerName="maas-api"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190074    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74e17325-bf76-4639-9475-b54f780ed429" containerName="maas-api"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190081    4926 cpu_manager.go:414] "containerMap: removing container" podUID="a977c9a1-fe79-4f13-8aa0-9a3484a28238" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190087    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a977c9a1-fe79-4f13-8aa0-9a3484a28238" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190095    4926 cpu_manager.go:414] "containerMap: removing container" podUID="69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190099    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190106    4926 cpu_manager.go:414] "containerMap: removing container" podUID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" containerName="main"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190110    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" containerName="main"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190164    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="69775e74-fc6d-4f0d-9f2e-b8e291ca00b0" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190172    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="a015eb6f-0f4c-4c57-b2aa-dc06e0fdf228" containerName="main"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190178    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ce8937f-8f05-4e63-85bb-5ee282b5367b" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190188    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="a07d7017-aef8-49fb-8de0-0b3cba42c240" containerName="maas-api"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190193    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="a65ad9de-668e-4600-8643-546a6ce94b99" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190198    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f0348bd-b29e-49eb-802a-dfcb95413b8a" containerName="main"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190203    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9f9a022-fa1c-4f08-ac54-cb751f629547" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190209    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9f51242-834d-4824-b9c9-05315f436249" containerName="maas-api"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190215    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="ade2ace8-0217-404c-9592-85576f16b085" containerName="main"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190219    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="0664091e-7353-433c-bd14-76c8b1b6d795" containerName="maas-api"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190224    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ce8937f-8f05-4e63-85bb-5ee282b5367b" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190229    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff3deeee-d307-4df0-8656-dcc3a13e060a" containerName="istio-proxy"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190235    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="74e17325-bf76-4639-9475-b54f780ed429" containerName="maas-api"
Oct 31 04:11:12.190238 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.190242    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="a977c9a1-fe79-4f13-8aa0-9a3484a28238" containerName="istio-proxy"
Oct 31 04:11:12.194604 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.194588    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6968-07b6b7"
Oct 31 04:11:12.243607 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.243571    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6968-07b6b7"]
Oct 31 04:11:12.248441 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.248414    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-2x2q5\""
Oct 31 04:11:12.295425 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.295390    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qvfp\" (UniqueName: \"kubernetes.io/projected/8e545c1f-0b6f-4e47-a54a-238137e641b7-kube-api-access-4qvfp\") pod \"test-curl-6968-07b6b7\" (UID: \"8e545c1f-0b6f-4e47-a54a-238137e641b7\") " pod="opendatahub/test-curl-6968-07b6b7"
Oct 31 04:11:12.396894 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.396853    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qvfp\" (UniqueName: \"kubernetes.io/projected/8e545c1f-0b6f-4e47-a54a-238137e641b7-kube-api-access-4qvfp\") pod \"test-curl-6968-07b6b7\" (UID: \"8e545c1f-0b6f-4e47-a54a-238137e641b7\") " pod="opendatahub/test-curl-6968-07b6b7"
Oct 31 04:11:12.459928 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.459898    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qvfp\" (UniqueName: \"kubernetes.io/projected/8e545c1f-0b6f-4e47-a54a-238137e641b7-kube-api-access-4qvfp\") pod \"test-curl-6968-07b6b7\" (UID: \"8e545c1f-0b6f-4e47-a54a-238137e641b7\") " pod="opendatahub/test-curl-6968-07b6b7"
Oct 31 04:11:12.505917 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.505888    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6968-07b6b7"
Oct 31 04:11:12.876818 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:11:12.876788    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e545c1f_0b6f_4e47_a54a_238137e641b7.slice/crio-3d42dc7ced93511d13ccffb9f14c8a9feb39404acd502a01687b00f753a43e10 WatchSource:0}: Error finding container 3d42dc7ced93511d13ccffb9f14c8a9feb39404acd502a01687b00f753a43e10: Status 404 returned error can't find the container with id 3d42dc7ced93511d13ccffb9f14c8a9feb39404acd502a01687b00f753a43e10
Oct 31 04:11:12.879482 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.879462    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-6968-07b6b7"]
Oct 31 04:11:12.989887 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.989848    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6968-07b6b7" event={"ID":"8e545c1f-0b6f-4e47-a54a-238137e641b7","Type":"ContainerStarted","Data":"8f396379d4507180497c1afa85a5e6ee4c81606922ec14f2943adde54348eae9"}
Oct 31 04:11:12.989887 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:12.989896    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-6968-07b6b7" event={"ID":"8e545c1f-0b6f-4e47-a54a-238137e641b7","Type":"ContainerStarted","Data":"3d42dc7ced93511d13ccffb9f14c8a9feb39404acd502a01687b00f753a43e10"}
Oct 31 04:11:13.140057 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:13.139958    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-6968-07b6b7" podStartSLOduration=1.139945005 podStartE2EDuration="1.139945005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:11:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:11:13.139169527 +0000 UTC m=+2569.724709786" watchObservedRunningTime="2026-10-31 04:11:13.139945005 +0000 UTC m=+2569.725485267"
Oct 31 04:11:14.582310 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:14.582249    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-6968-07b6b7"]
Oct 31 04:11:14.587774 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:14.587744    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-6968-07b6b7"]
Oct 31 04:11:14.998182 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:14.998145    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-6968-07b6b7" podUID="8e545c1f-0b6f-4e47-a54a-238137e641b7" containerName="test-curl-6968-07b6b7" containerID="cri-o://8f396379d4507180497c1afa85a5e6ee4c81606922ec14f2943adde54348eae9" gracePeriod=2
Oct 31 04:11:15.051019 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:11:15.050983    4926 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-6968-07b6b7\" is forbidden: User \"system:node:ip-10-0-130-19.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-130-19.us-west-2.compute.internal' and this object" podUID="8e545c1f-0b6f-4e47-a54a-238137e641b7" pod="opendatahub/test-curl-6968-07b6b7"
Oct 31 04:11:16.347460 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:16.347429    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"]
Oct 31 04:11:16.399024 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:16.398990    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"]
Oct 31 04:11:17.222842 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:17.222818    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6968-07b6b7"
Oct 31 04:11:17.346040 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:17.345959    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e545c1f-0b6f-4e47-a54a-238137e641b7-kube-api-access-4qvfp\" (UniqueName: \"kubernetes.io/projected/8e545c1f-0b6f-4e47-a54a-238137e641b7-kube-api-access-4qvfp\") pod \"8e545c1f-0b6f-4e47-a54a-238137e641b7\" (UID: \"8e545c1f-0b6f-4e47-a54a-238137e641b7\") "
Oct 31 04:11:17.348219 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:17.348200    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e545c1f-0b6f-4e47-a54a-238137e641b7-kube-api-access-4qvfp" pod "8e545c1f-0b6f-4e47-a54a-238137e641b7" (UID: "8e545c1f-0b6f-4e47-a54a-238137e641b7"). InnerVolumeSpecName "kube-api-access-4qvfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:17.447870 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:17.447832    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qvfp\" (UniqueName: \"kubernetes.io/projected/8e545c1f-0b6f-4e47-a54a-238137e641b7-kube-api-access-4qvfp\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:17.537531 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:17.537501    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e545c1f-0b6f-4e47-a54a-238137e641b7" path="/var/lib/kubelet/pods/8e545c1f-0b6f-4e47-a54a-238137e641b7/volumes"
Oct 31 04:11:18.015284 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:18.015224    4926 generic.go:356] "Generic (PLEG): container finished" podID="8e545c1f-0b6f-4e47-a54a-238137e641b7" containerID="8f396379d4507180497c1afa85a5e6ee4c81606922ec14f2943adde54348eae9" exitCode=137
Oct 31 04:11:18.015479 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:18.015292    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-6968-07b6b7"
Oct 31 04:11:18.015479 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:18.015315    4926 scope.go:122] "RemoveContainer" containerID="8f396379d4507180497c1afa85a5e6ee4c81606922ec14f2943adde54348eae9"
Oct 31 04:11:18.023648 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:18.023630    4926 scope.go:122] "RemoveContainer" containerID="8f396379d4507180497c1afa85a5e6ee4c81606922ec14f2943adde54348eae9"
Oct 31 04:11:18.023894 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:11:18.023872    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f396379d4507180497c1afa85a5e6ee4c81606922ec14f2943adde54348eae9\": container with ID starting with 8f396379d4507180497c1afa85a5e6ee4c81606922ec14f2943adde54348eae9 not found: ID does not exist" containerID="8f396379d4507180497c1afa85a5e6ee4c81606922ec14f2943adde54348eae9"
Oct 31 04:11:18.023959 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:18.023902    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f396379d4507180497c1afa85a5e6ee4c81606922ec14f2943adde54348eae9"} err="failed to get container status \"8f396379d4507180497c1afa85a5e6ee4c81606922ec14f2943adde54348eae9\": rpc error: code = NotFound desc = could not find container \"8f396379d4507180497c1afa85a5e6ee4c81606922ec14f2943adde54348eae9\": container with ID starting with 8f396379d4507180497c1afa85a5e6ee4c81606922ec14f2943adde54348eae9 not found: ID does not exist"
Oct 31 04:11:30.780617 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:30.780573    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"]
Oct 31 04:11:30.781193 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:30.780836    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t" podUID="170d646b-12d9-47eb-88db-232869e6854e" containerName="istio-proxy" containerID="cri-o://10d0d710a4e75b68d6bad90291c2e5253242ae5848a77c865601471bd11988b2" gracePeriod=30
Oct 31 04:11:31.367694 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.367653    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg" podUID="067339e5-0df5-421d-8b14-471a9b1a9735" containerName="main" containerID="cri-o://d5bd5a2191e0ed712c33d2bb0a24980e20245d6706163be9f493a31a406975d3" gracePeriod=45
Oct 31 04:11:31.419161 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.419125    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn" podUID="e20732a1-da5b-4e22-b861-6b03dfe62522" containerName="main" containerID="cri-o://ad0ad05a7736668eb55fb9d5b2847dd9e5220a541c0dfce59698c8723cfe31ed" gracePeriod=45
Oct 31 04:11:31.725290 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.725242    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 04:11:31.728586 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.728561    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.780499    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/067339e5-0df5-421d-8b14-471a9b1a9735-kube-api-access-k7rng\" (UniqueName: \"kubernetes.io/projected/067339e5-0df5-421d-8b14-471a9b1a9735-kube-api-access-k7rng\") pod \"067339e5-0df5-421d-8b14-471a9b1a9735\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") "
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.780596    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-tmp-dir\") pod \"e20732a1-da5b-4e22-b861-6b03dfe62522\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") "
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.780633    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-model-cache\") pod \"e20732a1-da5b-4e22-b861-6b03dfe62522\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") "
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.780661    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/067339e5-0df5-421d-8b14-471a9b1a9735-tls-certs\" (UniqueName: \"kubernetes.io/secret/067339e5-0df5-421d-8b14-471a9b1a9735-tls-certs\") pod \"067339e5-0df5-421d-8b14-471a9b1a9735\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") "
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.780698    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-model-cache\") pod \"067339e5-0df5-421d-8b14-471a9b1a9735\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") "
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.780773    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-tmp-dir\") pod \"067339e5-0df5-421d-8b14-471a9b1a9735\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") "
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.780827    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-dshm\") pod \"e20732a1-da5b-4e22-b861-6b03dfe62522\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") "
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.780869    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-dshm\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-dshm\") pod \"067339e5-0df5-421d-8b14-471a9b1a9735\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") "
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.780899    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-home\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-home\") pod \"067339e5-0df5-421d-8b14-471a9b1a9735\" (UID: \"067339e5-0df5-421d-8b14-471a9b1a9735\") "
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.780937    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-home\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-home\") pod \"e20732a1-da5b-4e22-b861-6b03dfe62522\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") "
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.780969    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e20732a1-da5b-4e22-b861-6b03dfe62522-tls-certs\" (UniqueName: \"kubernetes.io/secret/e20732a1-da5b-4e22-b861-6b03dfe62522-tls-certs\") pod \"e20732a1-da5b-4e22-b861-6b03dfe62522\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") "
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.781013    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-model-cache" pod "e20732a1-da5b-4e22-b861-6b03dfe62522" (UID: "e20732a1-da5b-4e22-b861-6b03dfe62522"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.781038    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e20732a1-da5b-4e22-b861-6b03dfe62522-kube-api-access-7r94s\" (UniqueName: \"kubernetes.io/projected/e20732a1-da5b-4e22-b861-6b03dfe62522-kube-api-access-7r94s\") pod \"e20732a1-da5b-4e22-b861-6b03dfe62522\" (UID: \"e20732a1-da5b-4e22-b861-6b03dfe62522\") "
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.781190    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-tmp-dir" pod "067339e5-0df5-421d-8b14-471a9b1a9735" (UID: "067339e5-0df5-421d-8b14-471a9b1a9735"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.781214    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-tmp-dir" pod "e20732a1-da5b-4e22-b861-6b03dfe62522" (UID: "e20732a1-da5b-4e22-b861-6b03dfe62522"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.781337    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-model-cache" pod "067339e5-0df5-421d-8b14-471a9b1a9735" (UID: "067339e5-0df5-421d-8b14-471a9b1a9735"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.781430    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-home" pod "067339e5-0df5-421d-8b14-471a9b1a9735" (UID: "067339e5-0df5-421d-8b14-471a9b1a9735"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.781477    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.781497    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.781511    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.781524    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.783626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.781540    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-home" pod "e20732a1-da5b-4e22-b861-6b03dfe62522" (UID: "e20732a1-da5b-4e22-b861-6b03dfe62522"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:31.784789 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.784386    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-dshm" pod "e20732a1-da5b-4e22-b861-6b03dfe62522" (UID: "e20732a1-da5b-4e22-b861-6b03dfe62522"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:31.786474 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.786448    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/067339e5-0df5-421d-8b14-471a9b1a9735-kube-api-access-k7rng" pod "067339e5-0df5-421d-8b14-471a9b1a9735" (UID: "067339e5-0df5-421d-8b14-471a9b1a9735"). InnerVolumeSpecName "kube-api-access-k7rng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:31.786474 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.786449    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-dshm" pod "067339e5-0df5-421d-8b14-471a9b1a9735" (UID: "067339e5-0df5-421d-8b14-471a9b1a9735"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:31.786652 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.786473    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e20732a1-da5b-4e22-b861-6b03dfe62522-kube-api-access-7r94s" pod "e20732a1-da5b-4e22-b861-6b03dfe62522" (UID: "e20732a1-da5b-4e22-b861-6b03dfe62522"). InnerVolumeSpecName "kube-api-access-7r94s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:31.786652 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.786597    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/067339e5-0df5-421d-8b14-471a9b1a9735-tls-certs" pod "067339e5-0df5-421d-8b14-471a9b1a9735" (UID: "067339e5-0df5-421d-8b14-471a9b1a9735"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:11:31.787803 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.787782    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20732a1-da5b-4e22-b861-6b03dfe62522-tls-certs" pod "e20732a1-da5b-4e22-b861-6b03dfe62522" (UID: "e20732a1-da5b-4e22-b861-6b03dfe62522"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:11:31.883039 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.882954    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7r94s\" (UniqueName: \"kubernetes.io/projected/e20732a1-da5b-4e22-b861-6b03dfe62522-kube-api-access-7r94s\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.883039 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.882981    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7rng\" (UniqueName: \"kubernetes.io/projected/067339e5-0df5-421d-8b14-471a9b1a9735-kube-api-access-k7rng\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.883039 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.882991    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/067339e5-0df5-421d-8b14-471a9b1a9735-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.883039 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.883000    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.883039 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.883008    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.883039 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.883015    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/067339e5-0df5-421d-8b14-471a9b1a9735-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.883039 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.883022    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e20732a1-da5b-4e22-b861-6b03dfe62522-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:31.883039 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:31.883031    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20732a1-da5b-4e22-b861-6b03dfe62522-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:32.082601 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.082572    4926 generic.go:356] "Generic (PLEG): container finished" podID="067339e5-0df5-421d-8b14-471a9b1a9735" containerID="d5bd5a2191e0ed712c33d2bb0a24980e20245d6706163be9f493a31a406975d3" exitCode=0
Oct 31 04:11:32.082785 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.082625    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"
Oct 31 04:11:32.082785 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.082657    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg" event={"ID":"067339e5-0df5-421d-8b14-471a9b1a9735","Type":"ContainerDied","Data":"d5bd5a2191e0ed712c33d2bb0a24980e20245d6706163be9f493a31a406975d3"}
Oct 31 04:11:32.082785 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.082702    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg" event={"ID":"067339e5-0df5-421d-8b14-471a9b1a9735","Type":"ContainerDied","Data":"e1616542656096d18c48186cfa6b271ccc74e3b8c59ad1990e94c2b14ae5f644"}
Oct 31 04:11:32.082785 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.082724    4926 scope.go:122] "RemoveContainer" containerID="d5bd5a2191e0ed712c33d2bb0a24980e20245d6706163be9f493a31a406975d3"
Oct 31 04:11:32.084188 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.084163    4926 generic.go:356] "Generic (PLEG): container finished" podID="e20732a1-da5b-4e22-b861-6b03dfe62522" containerID="ad0ad05a7736668eb55fb9d5b2847dd9e5220a541c0dfce59698c8723cfe31ed" exitCode=0
Oct 31 04:11:32.084299 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.084235    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn" event={"ID":"e20732a1-da5b-4e22-b861-6b03dfe62522","Type":"ContainerDied","Data":"ad0ad05a7736668eb55fb9d5b2847dd9e5220a541c0dfce59698c8723cfe31ed"}
Oct 31 04:11:32.084299 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.084263    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn" event={"ID":"e20732a1-da5b-4e22-b861-6b03dfe62522","Type":"ContainerDied","Data":"9e5b91e14bebad02fe07ffd0bd4ddafa41b79238695815bfea5a7d0ed35c11a5"}
Oct 31 04:11:32.084388 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.084375    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"
Oct 31 04:11:32.093527 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.092863    4926 scope.go:122] "RemoveContainer" containerID="d5bd5a2191e0ed712c33d2bb0a24980e20245d6706163be9f493a31a406975d3"
Oct 31 04:11:32.093527 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:11:32.093259    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5bd5a2191e0ed712c33d2bb0a24980e20245d6706163be9f493a31a406975d3\": container with ID starting with d5bd5a2191e0ed712c33d2bb0a24980e20245d6706163be9f493a31a406975d3 not found: ID does not exist" containerID="d5bd5a2191e0ed712c33d2bb0a24980e20245d6706163be9f493a31a406975d3"
Oct 31 04:11:32.093527 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.093323    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5bd5a2191e0ed712c33d2bb0a24980e20245d6706163be9f493a31a406975d3"} err="failed to get container status \"d5bd5a2191e0ed712c33d2bb0a24980e20245d6706163be9f493a31a406975d3\": rpc error: code = NotFound desc = could not find container \"d5bd5a2191e0ed712c33d2bb0a24980e20245d6706163be9f493a31a406975d3\": container with ID starting with d5bd5a2191e0ed712c33d2bb0a24980e20245d6706163be9f493a31a406975d3 not found: ID does not exist"
Oct 31 04:11:32.093527 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.093346    4926 scope.go:122] "RemoveContainer" containerID="ad0ad05a7736668eb55fb9d5b2847dd9e5220a541c0dfce59698c8723cfe31ed"
Oct 31 04:11:32.103224 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.103204    4926 scope.go:122] "RemoveContainer" containerID="ad0ad05a7736668eb55fb9d5b2847dd9e5220a541c0dfce59698c8723cfe31ed"
Oct 31 04:11:32.103520 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:11:32.103495    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ad0ad05a7736668eb55fb9d5b2847dd9e5220a541c0dfce59698c8723cfe31ed\": container with ID starting with ad0ad05a7736668eb55fb9d5b2847dd9e5220a541c0dfce59698c8723cfe31ed not found: ID does not exist" containerID="ad0ad05a7736668eb55fb9d5b2847dd9e5220a541c0dfce59698c8723cfe31ed"
Oct 31 04:11:32.103616 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.103523    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ad0ad05a7736668eb55fb9d5b2847dd9e5220a541c0dfce59698c8723cfe31ed"} err="failed to get container status \"ad0ad05a7736668eb55fb9d5b2847dd9e5220a541c0dfce59698c8723cfe31ed\": rpc error: code = NotFound desc = could not find container \"ad0ad05a7736668eb55fb9d5b2847dd9e5220a541c0dfce59698c8723cfe31ed\": container with ID starting with ad0ad05a7736668eb55fb9d5b2847dd9e5220a541c0dfce59698c8723cfe31ed not found: ID does not exist"
Oct 31 04:11:32.273253 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.273218    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"]
Oct 31 04:11:32.276423 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.276398    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/distinct-2-w5-e5b8f9-kserve-844d657b46-g2rrg"]
Oct 31 04:11:32.387754 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.387725    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"]
Oct 31 04:11:32.394605 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:32.394585    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-e5b8f9/premium-w5-e5b8f9-kserve-df79994f8-54pfn"]
Oct 31 04:11:33.537775 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:33.537738    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="067339e5-0df5-421d-8b14-471a9b1a9735" path="/var/lib/kubelet/pods/067339e5-0df5-421d-8b14-471a9b1a9735/volumes"
Oct 31 04:11:33.538164 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:33.538132    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e20732a1-da5b-4e22-b861-6b03dfe62522" path="/var/lib/kubelet/pods/e20732a1-da5b-4e22-b861-6b03dfe62522/volumes"
Oct 31 04:11:34.555656 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:34.555619    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-qk79t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 31 04:11:34.556056 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:34.555684    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t" podUID="170d646b-12d9-47eb-88db-232869e6854e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 31 04:11:35.311984 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:11:35.311938    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:11:36.030975 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.030950    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 04:11:36.106987 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.106900    4926 generic.go:356] "Generic (PLEG): container finished" podID="170d646b-12d9-47eb-88db-232869e6854e" containerID="10d0d710a4e75b68d6bad90291c2e5253242ae5848a77c865601471bd11988b2" exitCode=0
Oct 31 04:11:36.106987 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.106960    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"
Oct 31 04:11:36.107169 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.106993    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t" event={"ID":"170d646b-12d9-47eb-88db-232869e6854e","Type":"ContainerDied","Data":"10d0d710a4e75b68d6bad90291c2e5253242ae5848a77c865601471bd11988b2"}
Oct 31 04:11:36.107169 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.107042    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t" event={"ID":"170d646b-12d9-47eb-88db-232869e6854e","Type":"ContainerDied","Data":"3dbc46a0966c4cf9588743d9b9584d405bbf6ed059f44991b5690cb90dab9c40"}
Oct 31 04:11:36.107169 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.107066    4926 scope.go:122] "RemoveContainer" containerID="10d0d710a4e75b68d6bad90291c2e5253242ae5848a77c865601471bd11988b2"
Oct 31 04:11:36.115880 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.115863    4926 scope.go:122] "RemoveContainer" containerID="10d0d710a4e75b68d6bad90291c2e5253242ae5848a77c865601471bd11988b2"
Oct 31 04:11:36.116137 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:11:36.116116    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10d0d710a4e75b68d6bad90291c2e5253242ae5848a77c865601471bd11988b2\": container with ID starting with 10d0d710a4e75b68d6bad90291c2e5253242ae5848a77c865601471bd11988b2 not found: ID does not exist" containerID="10d0d710a4e75b68d6bad90291c2e5253242ae5848a77c865601471bd11988b2"
Oct 31 04:11:36.116188 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.116146    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10d0d710a4e75b68d6bad90291c2e5253242ae5848a77c865601471bd11988b2"} err="failed to get container status \"10d0d710a4e75b68d6bad90291c2e5253242ae5848a77c865601471bd11988b2\": rpc error: code = NotFound desc = could not find container \"10d0d710a4e75b68d6bad90291c2e5253242ae5848a77c865601471bd11988b2\": container with ID starting with 10d0d710a4e75b68d6bad90291c2e5253242ae5848a77c865601471bd11988b2 not found: ID does not exist"
Oct 31 04:11:36.123474 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.123455    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/170d646b-12d9-47eb-88db-232869e6854e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/170d646b-12d9-47eb-88db-232869e6854e-istiod-ca-cert\") pod \"170d646b-12d9-47eb-88db-232869e6854e\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") "
Oct 31 04:11:36.123538 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.123492    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/170d646b-12d9-47eb-88db-232869e6854e-istio-token\" (UniqueName: \"kubernetes.io/projected/170d646b-12d9-47eb-88db-232869e6854e-istio-token\") pod \"170d646b-12d9-47eb-88db-232869e6854e\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") "
Oct 31 04:11:36.123538 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.123524    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-istio-data\") pod \"170d646b-12d9-47eb-88db-232869e6854e\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") "
Oct 31 04:11:36.123606 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.123558    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-credential-socket\") pod \"170d646b-12d9-47eb-88db-232869e6854e\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") "
Oct 31 04:11:36.123639 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.123616    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/170d646b-12d9-47eb-88db-232869e6854e-kube-api-access-5gt6l\" (UniqueName: \"kubernetes.io/projected/170d646b-12d9-47eb-88db-232869e6854e-kube-api-access-5gt6l\") pod \"170d646b-12d9-47eb-88db-232869e6854e\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") "
Oct 31 04:11:36.123682 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.123647    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/170d646b-12d9-47eb-88db-232869e6854e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/170d646b-12d9-47eb-88db-232869e6854e-istio-podinfo\") pod \"170d646b-12d9-47eb-88db-232869e6854e\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") "
Oct 31 04:11:36.123730 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.123680    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-workload-socket\") pod \"170d646b-12d9-47eb-88db-232869e6854e\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") "
Oct 31 04:11:36.123730 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.123718    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-istio-envoy\") pod \"170d646b-12d9-47eb-88db-232869e6854e\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") "
Oct 31 04:11:36.123865 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.123846    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-istio-data" pod "170d646b-12d9-47eb-88db-232869e6854e" (UID: "170d646b-12d9-47eb-88db-232869e6854e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.123865 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.123857    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-workload-certs\") pod \"170d646b-12d9-47eb-88db-232869e6854e\" (UID: \"170d646b-12d9-47eb-88db-232869e6854e\") "
Oct 31 04:11:36.123982 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.123939    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-credential-socket" pod "170d646b-12d9-47eb-88db-232869e6854e" (UID: "170d646b-12d9-47eb-88db-232869e6854e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.123982 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.123856    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/170d646b-12d9-47eb-88db-232869e6854e-istiod-ca-cert" pod "170d646b-12d9-47eb-88db-232869e6854e" (UID: "170d646b-12d9-47eb-88db-232869e6854e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:11:36.124108 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.124066    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-workload-socket" pod "170d646b-12d9-47eb-88db-232869e6854e" (UID: "170d646b-12d9-47eb-88db-232869e6854e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.124163 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.124125    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-workload-certs" pod "170d646b-12d9-47eb-88db-232869e6854e" (UID: "170d646b-12d9-47eb-88db-232869e6854e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.124255 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.124241    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.124497 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.124259    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.124497 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.124304    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/170d646b-12d9-47eb-88db-232869e6854e-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.124497 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.124317    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.124497 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.124331    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.125936 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.125904    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/170d646b-12d9-47eb-88db-232869e6854e-kube-api-access-5gt6l" pod "170d646b-12d9-47eb-88db-232869e6854e" (UID: "170d646b-12d9-47eb-88db-232869e6854e"). InnerVolumeSpecName "kube-api-access-5gt6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:36.125936 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.125917    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/170d646b-12d9-47eb-88db-232869e6854e-istio-token" pod "170d646b-12d9-47eb-88db-232869e6854e" (UID: "170d646b-12d9-47eb-88db-232869e6854e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:36.125936 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.125923    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/170d646b-12d9-47eb-88db-232869e6854e-istio-podinfo" pod "170d646b-12d9-47eb-88db-232869e6854e" (UID: "170d646b-12d9-47eb-88db-232869e6854e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:11:36.126317 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.126256    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-istio-envoy" pod "170d646b-12d9-47eb-88db-232869e6854e" (UID: "170d646b-12d9-47eb-88db-232869e6854e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:36.225600 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.225566    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/170d646b-12d9-47eb-88db-232869e6854e-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.225600 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.225595    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gt6l\" (UniqueName: \"kubernetes.io/projected/170d646b-12d9-47eb-88db-232869e6854e-kube-api-access-5gt6l\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.225600 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.225607    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/170d646b-12d9-47eb-88db-232869e6854e-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.225907 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.225615    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/170d646b-12d9-47eb-88db-232869e6854e-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:36.535184 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.535153    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"]
Oct 31 04:11:36.543015 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:36.542984    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-qk79t"]
Oct 31 04:11:37.537792 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:37.537760    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="170d646b-12d9-47eb-88db-232869e6854e" path="/var/lib/kubelet/pods/170d646b-12d9-47eb-88db-232869e6854e/volumes"
Oct 31 04:11:38.343146 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:38.343112    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"]
Oct 31 04:11:45.865049 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:45.865013    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"]
Oct 31 04:11:45.865540 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:45.865232    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerName="istio-proxy" containerID="cri-o://86ace4b78496d8eb24077904d2a339dbc61dd08c60096983dcc94a10ed3a35d5" gracePeriod=30
Oct 31 04:11:51.124356 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.124328    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 04:11:51.183219 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.183143    4926 generic.go:356] "Generic (PLEG): container finished" podID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerID="86ace4b78496d8eb24077904d2a339dbc61dd08c60096983dcc94a10ed3a35d5" exitCode=0
Oct 31 04:11:51.183219 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.183193    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"
Oct 31 04:11:51.183219 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.183210    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" event={"ID":"d41233ae-9c3e-466a-aa39-075d14ba5e0f","Type":"ContainerDied","Data":"86ace4b78496d8eb24077904d2a339dbc61dd08c60096983dcc94a10ed3a35d5"}
Oct 31 04:11:51.183449 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.183250    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw" event={"ID":"d41233ae-9c3e-466a-aa39-075d14ba5e0f","Type":"ContainerDied","Data":"dc476f85b109d60d12def75090137c12b343d569e73bffb3b5e31717a26bb42f"}
Oct 31 04:11:51.183449 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.183297    4926 scope.go:122] "RemoveContainer" containerID="86ace4b78496d8eb24077904d2a339dbc61dd08c60096983dcc94a10ed3a35d5"
Oct 31 04:11:51.191788 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.191770    4926 scope.go:122] "RemoveContainer" containerID="86ace4b78496d8eb24077904d2a339dbc61dd08c60096983dcc94a10ed3a35d5"
Oct 31 04:11:51.192067 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:11:51.192038    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86ace4b78496d8eb24077904d2a339dbc61dd08c60096983dcc94a10ed3a35d5\": container with ID starting with 86ace4b78496d8eb24077904d2a339dbc61dd08c60096983dcc94a10ed3a35d5 not found: ID does not exist" containerID="86ace4b78496d8eb24077904d2a339dbc61dd08c60096983dcc94a10ed3a35d5"
Oct 31 04:11:51.192164 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.192064    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86ace4b78496d8eb24077904d2a339dbc61dd08c60096983dcc94a10ed3a35d5"} err="failed to get container status \"86ace4b78496d8eb24077904d2a339dbc61dd08c60096983dcc94a10ed3a35d5\": rpc error: code = NotFound desc = could not find container \"86ace4b78496d8eb24077904d2a339dbc61dd08c60096983dcc94a10ed3a35d5\": container with ID starting with 86ace4b78496d8eb24077904d2a339dbc61dd08c60096983dcc94a10ed3a35d5 not found: ID does not exist"
Oct 31 04:11:51.280449 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.280420    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-workload-socket\") pod \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") "
Oct 31 04:11:51.280630 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.280468    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-token\" (UniqueName: \"kubernetes.io/projected/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-token\") pod \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") "
Oct 31 04:11:51.280630 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.280501    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d41233ae-9c3e-466a-aa39-075d14ba5e0f-kube-api-access-5bfk8\" (UniqueName: \"kubernetes.io/projected/d41233ae-9c3e-466a-aa39-075d14ba5e0f-kube-api-access-5bfk8\") pod \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") "
Oct 31 04:11:51.280630 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.280518    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-podinfo\") pod \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") "
Oct 31 04:11:51.280630 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.280552    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-data\") pod \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") "
Oct 31 04:11:51.280630 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.280568    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-credential-socket\") pod \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") "
Oct 31 04:11:51.280879 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.280636    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istiod-ca-cert\") pod \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") "
Oct 31 04:11:51.280879 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.280701    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-envoy\") pod \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") "
Oct 31 04:11:51.280879 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.280732    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-workload-certs\") pod \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\" (UID: \"d41233ae-9c3e-466a-aa39-075d14ba5e0f\") "
Oct 31 04:11:51.280879 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.280746    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-workload-socket" pod "d41233ae-9c3e-466a-aa39-075d14ba5e0f" (UID: "d41233ae-9c3e-466a-aa39-075d14ba5e0f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:51.281088 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.280977    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-data" pod "d41233ae-9c3e-466a-aa39-075d14ba5e0f" (UID: "d41233ae-9c3e-466a-aa39-075d14ba5e0f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:51.281088 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.281037    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.281088 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.281080    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istiod-ca-cert" pod "d41233ae-9c3e-466a-aa39-075d14ba5e0f" (UID: "d41233ae-9c3e-466a-aa39-075d14ba5e0f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:11:51.281240 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.281212    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-workload-certs" pod "d41233ae-9c3e-466a-aa39-075d14ba5e0f" (UID: "d41233ae-9c3e-466a-aa39-075d14ba5e0f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:51.281341 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.281298    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-credential-socket" pod "d41233ae-9c3e-466a-aa39-075d14ba5e0f" (UID: "d41233ae-9c3e-466a-aa39-075d14ba5e0f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:51.283019 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.282988    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d41233ae-9c3e-466a-aa39-075d14ba5e0f-kube-api-access-5bfk8" pod "d41233ae-9c3e-466a-aa39-075d14ba5e0f" (UID: "d41233ae-9c3e-466a-aa39-075d14ba5e0f"). InnerVolumeSpecName "kube-api-access-5bfk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:51.283146 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.283031    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-envoy" pod "d41233ae-9c3e-466a-aa39-075d14ba5e0f" (UID: "d41233ae-9c3e-466a-aa39-075d14ba5e0f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:51.283146 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.283041    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-token" pod "d41233ae-9c3e-466a-aa39-075d14ba5e0f" (UID: "d41233ae-9c3e-466a-aa39-075d14ba5e0f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:51.283146 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.283108    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-podinfo" pod "d41233ae-9c3e-466a-aa39-075d14ba5e0f" (UID: "d41233ae-9c3e-466a-aa39-075d14ba5e0f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:11:51.381932 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.381903    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bfk8\" (UniqueName: \"kubernetes.io/projected/d41233ae-9c3e-466a-aa39-075d14ba5e0f-kube-api-access-5bfk8\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.381932 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.381930    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.382116 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.381940    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.382116 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.381949    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.382116 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.381973    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.382116 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.381983    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.382116 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.381992    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d41233ae-9c3e-466a-aa39-075d14ba5e0f-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.382116 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.382001    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d41233ae-9c3e-466a-aa39-075d14ba5e0f-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:51.605840 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.605808    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"]
Oct 31 04:11:51.611360 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:51.611334    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-e5b8f9-openshift-default-5f9ddc7f7c-pcwbw"]
Oct 31 04:11:53.363369 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.363292    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn" podUID="e430bd98-6cc2-4b3a-a521-71a3e2547fdc" containerName="main" containerID="cri-o://64eba575b8823370f88377a9eff06730c784bae28401f5c56589ced75af40cf5" gracePeriod=45
Oct 31 04:11:53.538848 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.538818    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" path="/var/lib/kubelet/pods/d41233ae-9c3e-466a-aa39-075d14ba5e0f/volumes"
Oct 31 04:11:53.615244 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.615187    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:11:53.703835 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.703801    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-dshm\") pod \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") "
Oct 31 04:11:53.703998 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.703850    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-home\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-home\") pod \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") "
Oct 31 04:11:53.703998 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.703938    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-kube-api-access-4t556\" (UniqueName: \"kubernetes.io/projected/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-kube-api-access-4t556\") pod \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") "
Oct 31 04:11:53.703998 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.703969    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-model-cache\") pod \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") "
Oct 31 04:11:53.704161 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.704025    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-tmp-dir\") pod \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") "
Oct 31 04:11:53.704161 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.704053    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-tls-certs\" (UniqueName: \"kubernetes.io/secret/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-tls-certs\") pod \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\" (UID: \"e430bd98-6cc2-4b3a-a521-71a3e2547fdc\") "
Oct 31 04:11:53.704250 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.704162    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-home" pod "e430bd98-6cc2-4b3a-a521-71a3e2547fdc" (UID: "e430bd98-6cc2-4b3a-a521-71a3e2547fdc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:53.704334 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.704242    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-model-cache" pod "e430bd98-6cc2-4b3a-a521-71a3e2547fdc" (UID: "e430bd98-6cc2-4b3a-a521-71a3e2547fdc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:53.704334 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.704302    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-tmp-dir" pod "e430bd98-6cc2-4b3a-a521-71a3e2547fdc" (UID: "e430bd98-6cc2-4b3a-a521-71a3e2547fdc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:53.704493 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.704471    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:53.704493 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.704492    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:53.704863 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.704503    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:53.706348 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.706322    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-dshm" pod "e430bd98-6cc2-4b3a-a521-71a3e2547fdc" (UID: "e430bd98-6cc2-4b3a-a521-71a3e2547fdc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:11:53.706442 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.706356    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-tls-certs" pod "e430bd98-6cc2-4b3a-a521-71a3e2547fdc" (UID: "e430bd98-6cc2-4b3a-a521-71a3e2547fdc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:11:53.706442 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.706379    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-kube-api-access-4t556" pod "e430bd98-6cc2-4b3a-a521-71a3e2547fdc" (UID: "e430bd98-6cc2-4b3a-a521-71a3e2547fdc"). InnerVolumeSpecName "kube-api-access-4t556". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:11:53.805187 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.805158    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:53.805187 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.805185    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t556\" (UniqueName: \"kubernetes.io/projected/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-kube-api-access-4t556\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:53.805403 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:53.805196    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e430bd98-6cc2-4b3a-a521-71a3e2547fdc-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:11:54.198825 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:54.198791    4926 generic.go:356] "Generic (PLEG): container finished" podID="e430bd98-6cc2-4b3a-a521-71a3e2547fdc" containerID="64eba575b8823370f88377a9eff06730c784bae28401f5c56589ced75af40cf5" exitCode=0
Oct 31 04:11:54.198971 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:54.198859    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"
Oct 31 04:11:54.198971 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:54.198876    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn" event={"ID":"e430bd98-6cc2-4b3a-a521-71a3e2547fdc","Type":"ContainerDied","Data":"64eba575b8823370f88377a9eff06730c784bae28401f5c56589ced75af40cf5"}
Oct 31 04:11:54.198971 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:54.198932    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn" event={"ID":"e430bd98-6cc2-4b3a-a521-71a3e2547fdc","Type":"ContainerDied","Data":"88635d6e1bbad629ff47e45b7dd663c705f9bf4ab0e44c4c69b67972254c41d4"}
Oct 31 04:11:54.198971 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:54.198957    4926 scope.go:122] "RemoveContainer" containerID="64eba575b8823370f88377a9eff06730c784bae28401f5c56589ced75af40cf5"
Oct 31 04:11:54.209694 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:54.209672    4926 scope.go:122] "RemoveContainer" containerID="64eba575b8823370f88377a9eff06730c784bae28401f5c56589ced75af40cf5"
Oct 31 04:11:54.210305 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:11:54.210034    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64eba575b8823370f88377a9eff06730c784bae28401f5c56589ced75af40cf5\": container with ID starting with 64eba575b8823370f88377a9eff06730c784bae28401f5c56589ced75af40cf5 not found: ID does not exist" containerID="64eba575b8823370f88377a9eff06730c784bae28401f5c56589ced75af40cf5"
Oct 31 04:11:54.210305 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:54.210065    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64eba575b8823370f88377a9eff06730c784bae28401f5c56589ced75af40cf5"} err="failed to get container status \"64eba575b8823370f88377a9eff06730c784bae28401f5c56589ced75af40cf5\": rpc error: code = NotFound desc = could not find container \"64eba575b8823370f88377a9eff06730c784bae28401f5c56589ced75af40cf5\": container with ID starting with 64eba575b8823370f88377a9eff06730c784bae28401f5c56589ced75af40cf5 not found: ID does not exist"
Oct 31 04:11:54.367479 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:54.367444    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"]
Oct 31 04:11:54.371397 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:54.371373    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-6e81aea5/e2e-lifecycle-model-6e81aea5-kserve-79fc496d9c-gjjjn"]
Oct 31 04:11:55.538376 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:11:55.538342    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e430bd98-6cc2-4b3a-a521-71a3e2547fdc" path="/var/lib/kubelet/pods/e430bd98-6cc2-4b3a-a521-71a3e2547fdc/volumes"
Oct 31 04:12:05.311770 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:12:05.311737    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:12:27.950186 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:27.950147    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"]
Oct 31 04:12:28.001349 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:28.001308    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"]
Oct 31 04:12:28.002715 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:28.002689    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"]
Oct 31 04:12:35.312804 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:12:35.312763    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:12:42.970691 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:42.970638    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerName="main" containerID="cri-o://6e06f93869ac36c4449d03568997c1a54322e218503a8513c9a8bdc3007e47aa" gracePeriod=45
Oct 31 04:12:43.021325 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.021252    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l" podUID="a5f198d8-39a2-4c72-87de-516e077d47bd" containerName="main" containerID="cri-o://adcbc79f85d386554141f2ff73561735fb0ac52fb6fa1da2ea1d1fe90d4337a8" gracePeriod=45
Oct 31 04:12:43.021933 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.021886    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j" podUID="f97017f6-52c4-4d27-9aa7-5045b9ac85ee" containerName="main" containerID="cri-o://18bbb12ec7ffebd2133be31b7aaaf918e3d4221354019bf4a216768570e98ba4" gracePeriod=45
Oct 31 04:12:43.284619 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.284593    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 04:12:43.309612 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.309588    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 04:12:43.312412 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.312396    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 04:12:43.378179 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378148    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-dshm\") pod \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") "
Oct 31 04:12:43.378384 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378190    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-model-cache\") pod \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") "
Oct 31 04:12:43.378384 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378225    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-tmp-dir\") pod \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") "
Oct 31 04:12:43.378384 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378250    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-tls-certs\" (UniqueName: \"kubernetes.io/secret/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-tls-certs\") pod \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") "
Oct 31 04:12:43.378384 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378293    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-tls-certs\" (UniqueName: \"kubernetes.io/secret/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-tls-certs\") pod \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") "
Oct 31 04:12:43.378384 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378319    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a5f198d8-39a2-4c72-87de-516e077d47bd-tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f198d8-39a2-4c72-87de-516e077d47bd-tls-certs\") pod \"a5f198d8-39a2-4c72-87de-516e077d47bd\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") "
Oct 31 04:12:43.378384 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378344    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5f198d8-39a2-4c72-87de-516e077d47bd-kube-api-access-tj7gn\" (UniqueName: \"kubernetes.io/projected/a5f198d8-39a2-4c72-87de-516e077d47bd-kube-api-access-tj7gn\") pod \"a5f198d8-39a2-4c72-87de-516e077d47bd\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") "
Oct 31 04:12:43.378384 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378381    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-home\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-home\") pod \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") "
Oct 31 04:12:43.378820 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378406    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-kube-api-access-8cfgh\" (UniqueName: \"kubernetes.io/projected/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-kube-api-access-8cfgh\") pod \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") "
Oct 31 04:12:43.378820 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378460    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-home\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-home\") pod \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") "
Oct 31 04:12:43.378820 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378498    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-model-cache\") pod \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") "
Oct 31 04:12:43.378820 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378512    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-model-cache" pod "f97017f6-52c4-4d27-9aa7-5045b9ac85ee" (UID: "f97017f6-52c4-4d27-9aa7-5045b9ac85ee"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:43.378820 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378559    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-tmp-dir" pod "e5699efa-62d9-4cb3-b53b-bd83adff2f1d" (UID: "e5699efa-62d9-4cb3-b53b-bd83adff2f1d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:43.378820 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378586    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-dshm\") pod \"a5f198d8-39a2-4c72-87de-516e077d47bd\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") "
Oct 31 04:12:43.378820 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378618    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-model-cache\") pod \"a5f198d8-39a2-4c72-87de-516e077d47bd\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") "
Oct 31 04:12:43.378820 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378659    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-dshm\") pod \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") "
Oct 31 04:12:43.378820 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378695    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-home\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-home\") pod \"a5f198d8-39a2-4c72-87de-516e077d47bd\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") "
Oct 31 04:12:43.378820 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378741    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-tmp-dir\") pod \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\" (UID: \"f97017f6-52c4-4d27-9aa7-5045b9ac85ee\") "
Oct 31 04:12:43.378820 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378789    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-kube-api-access-rdn8x\" (UniqueName: \"kubernetes.io/projected/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-kube-api-access-rdn8x\") pod \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\" (UID: \"e5699efa-62d9-4cb3-b53b-bd83adff2f1d\") "
Oct 31 04:12:43.378820 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378820    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-tmp-dir\") pod \"a5f198d8-39a2-4c72-87de-516e077d47bd\" (UID: \"a5f198d8-39a2-4c72-87de-516e077d47bd\") "
Oct 31 04:12:43.379443 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.378851    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-model-cache" pod "e5699efa-62d9-4cb3-b53b-bd83adff2f1d" (UID: "e5699efa-62d9-4cb3-b53b-bd83adff2f1d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:43.379443 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.379170    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.379443 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.379188    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.379443 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.379202    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.379654 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.379456    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-tmp-dir" pod "a5f198d8-39a2-4c72-87de-516e077d47bd" (UID: "a5f198d8-39a2-4c72-87de-516e077d47bd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:43.379654 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.379552    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-home" pod "f97017f6-52c4-4d27-9aa7-5045b9ac85ee" (UID: "f97017f6-52c4-4d27-9aa7-5045b9ac85ee"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:43.379654 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.379644    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-home" pod "a5f198d8-39a2-4c72-87de-516e077d47bd" (UID: "a5f198d8-39a2-4c72-87de-516e077d47bd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:43.379825 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.379786    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-home" pod "e5699efa-62d9-4cb3-b53b-bd83adff2f1d" (UID: "e5699efa-62d9-4cb3-b53b-bd83adff2f1d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:43.379881 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.379828    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-tmp-dir" pod "f97017f6-52c4-4d27-9aa7-5045b9ac85ee" (UID: "f97017f6-52c4-4d27-9aa7-5045b9ac85ee"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:43.380015 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.379976    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-model-cache" pod "a5f198d8-39a2-4c72-87de-516e077d47bd" (UID: "a5f198d8-39a2-4c72-87de-516e077d47bd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:43.381413 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.381373    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-tls-certs" pod "e5699efa-62d9-4cb3-b53b-bd83adff2f1d" (UID: "e5699efa-62d9-4cb3-b53b-bd83adff2f1d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:12:43.381931 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.381905    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-tls-certs" pod "f97017f6-52c4-4d27-9aa7-5045b9ac85ee" (UID: "f97017f6-52c4-4d27-9aa7-5045b9ac85ee"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:12:43.382041 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.381951    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-dshm" pod "f97017f6-52c4-4d27-9aa7-5045b9ac85ee" (UID: "f97017f6-52c4-4d27-9aa7-5045b9ac85ee"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:43.382377 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.382293    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5f198d8-39a2-4c72-87de-516e077d47bd-kube-api-access-tj7gn" pod "a5f198d8-39a2-4c72-87de-516e077d47bd" (UID: "a5f198d8-39a2-4c72-87de-516e077d47bd"). InnerVolumeSpecName "kube-api-access-tj7gn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:12:43.382561 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.382542    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5f198d8-39a2-4c72-87de-516e077d47bd-tls-certs" pod "a5f198d8-39a2-4c72-87de-516e077d47bd" (UID: "a5f198d8-39a2-4c72-87de-516e077d47bd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:12:43.382561 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.382545    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-kube-api-access-8cfgh" pod "f97017f6-52c4-4d27-9aa7-5045b9ac85ee" (UID: "f97017f6-52c4-4d27-9aa7-5045b9ac85ee"). InnerVolumeSpecName "kube-api-access-8cfgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:12:43.383603 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.383575    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-dshm" pod "a5f198d8-39a2-4c72-87de-516e077d47bd" (UID: "a5f198d8-39a2-4c72-87de-516e077d47bd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:43.383603 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.383580    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-dshm" pod "e5699efa-62d9-4cb3-b53b-bd83adff2f1d" (UID: "e5699efa-62d9-4cb3-b53b-bd83adff2f1d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:12:43.383998 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.383982    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-kube-api-access-rdn8x" pod "e5699efa-62d9-4cb3-b53b-bd83adff2f1d" (UID: "e5699efa-62d9-4cb3-b53b-bd83adff2f1d"). InnerVolumeSpecName "kube-api-access-rdn8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:12:43.433750 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.433714    4926 generic.go:356] "Generic (PLEG): container finished" podID="a5f198d8-39a2-4c72-87de-516e077d47bd" containerID="adcbc79f85d386554141f2ff73561735fb0ac52fb6fa1da2ea1d1fe90d4337a8" exitCode=0
Oct 31 04:12:43.433928 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.433778    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"
Oct 31 04:12:43.433928 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.433798    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l" event={"ID":"a5f198d8-39a2-4c72-87de-516e077d47bd","Type":"ContainerDied","Data":"adcbc79f85d386554141f2ff73561735fb0ac52fb6fa1da2ea1d1fe90d4337a8"}
Oct 31 04:12:43.433928 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.433847    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l" event={"ID":"a5f198d8-39a2-4c72-87de-516e077d47bd","Type":"ContainerDied","Data":"13a2cb63d4ef4b272d4533ca7ffb290ee70d5cfa34c8f936ef251dcde505ddba"}
Oct 31 04:12:43.433928 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.433865    4926 scope.go:122] "RemoveContainer" containerID="adcbc79f85d386554141f2ff73561735fb0ac52fb6fa1da2ea1d1fe90d4337a8"
Oct 31 04:12:43.435324 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.435304    4926 generic.go:356] "Generic (PLEG): container finished" podID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerID="6e06f93869ac36c4449d03568997c1a54322e218503a8513c9a8bdc3007e47aa" exitCode=0
Oct 31 04:12:43.435500 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.435340    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"
Oct 31 04:12:43.435500 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.435388    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" event={"ID":"e5699efa-62d9-4cb3-b53b-bd83adff2f1d","Type":"ContainerDied","Data":"6e06f93869ac36c4449d03568997c1a54322e218503a8513c9a8bdc3007e47aa"}
Oct 31 04:12:43.435500 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.435421    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7" event={"ID":"e5699efa-62d9-4cb3-b53b-bd83adff2f1d","Type":"ContainerDied","Data":"c2ab9f01aef83a5d0869bbc5b268b43733666ddf1d25620de1fa15024ab77160"}
Oct 31 04:12:43.436833 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.436813    4926 generic.go:356] "Generic (PLEG): container finished" podID="f97017f6-52c4-4d27-9aa7-5045b9ac85ee" containerID="18bbb12ec7ffebd2133be31b7aaaf918e3d4221354019bf4a216768570e98ba4" exitCode=0
Oct 31 04:12:43.436937 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.436883    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j" event={"ID":"f97017f6-52c4-4d27-9aa7-5045b9ac85ee","Type":"ContainerDied","Data":"18bbb12ec7ffebd2133be31b7aaaf918e3d4221354019bf4a216768570e98ba4"}
Oct 31 04:12:43.436937 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.436901    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j" event={"ID":"f97017f6-52c4-4d27-9aa7-5045b9ac85ee","Type":"ContainerDied","Data":"a3a1bd43c961326de78f60c8f1b40371474aeecbb8a3319c8fff295beb22d054"}
Oct 31 04:12:43.437060 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.436948    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"
Oct 31 04:12:43.447492 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.447476    4926 scope.go:122] "RemoveContainer" containerID="adcbc79f85d386554141f2ff73561735fb0ac52fb6fa1da2ea1d1fe90d4337a8"
Oct 31 04:12:43.456493 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:12:43.456442    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adcbc79f85d386554141f2ff73561735fb0ac52fb6fa1da2ea1d1fe90d4337a8\": container with ID starting with adcbc79f85d386554141f2ff73561735fb0ac52fb6fa1da2ea1d1fe90d4337a8 not found: ID does not exist" containerID="adcbc79f85d386554141f2ff73561735fb0ac52fb6fa1da2ea1d1fe90d4337a8"
Oct 31 04:12:43.456612 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.456490    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adcbc79f85d386554141f2ff73561735fb0ac52fb6fa1da2ea1d1fe90d4337a8"} err="failed to get container status \"adcbc79f85d386554141f2ff73561735fb0ac52fb6fa1da2ea1d1fe90d4337a8\": rpc error: code = NotFound desc = could not find container \"adcbc79f85d386554141f2ff73561735fb0ac52fb6fa1da2ea1d1fe90d4337a8\": container with ID starting with adcbc79f85d386554141f2ff73561735fb0ac52fb6fa1da2ea1d1fe90d4337a8 not found: ID does not exist"
Oct 31 04:12:43.456612 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.456518    4926 scope.go:122] "RemoveContainer" containerID="6e06f93869ac36c4449d03568997c1a54322e218503a8513c9a8bdc3007e47aa"
Oct 31 04:12:43.464821 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.464803    4926 scope.go:122] "RemoveContainer" containerID="6e06f93869ac36c4449d03568997c1a54322e218503a8513c9a8bdc3007e47aa"
Oct 31 04:12:43.465073 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:12:43.465053    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e06f93869ac36c4449d03568997c1a54322e218503a8513c9a8bdc3007e47aa\": container with ID starting with 6e06f93869ac36c4449d03568997c1a54322e218503a8513c9a8bdc3007e47aa not found: ID does not exist" containerID="6e06f93869ac36c4449d03568997c1a54322e218503a8513c9a8bdc3007e47aa"
Oct 31 04:12:43.465150 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.465085    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e06f93869ac36c4449d03568997c1a54322e218503a8513c9a8bdc3007e47aa"} err="failed to get container status \"6e06f93869ac36c4449d03568997c1a54322e218503a8513c9a8bdc3007e47aa\": rpc error: code = NotFound desc = could not find container \"6e06f93869ac36c4449d03568997c1a54322e218503a8513c9a8bdc3007e47aa\": container with ID starting with 6e06f93869ac36c4449d03568997c1a54322e218503a8513c9a8bdc3007e47aa not found: ID does not exist"
Oct 31 04:12:43.465150 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.465108    4926 scope.go:122] "RemoveContainer" containerID="18bbb12ec7ffebd2133be31b7aaaf918e3d4221354019bf4a216768570e98ba4"
Oct 31 04:12:43.472225 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.472204    4926 scope.go:122] "RemoveContainer" containerID="18bbb12ec7ffebd2133be31b7aaaf918e3d4221354019bf4a216768570e98ba4"
Oct 31 04:12:43.472479 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:12:43.472458    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18bbb12ec7ffebd2133be31b7aaaf918e3d4221354019bf4a216768570e98ba4\": container with ID starting with 18bbb12ec7ffebd2133be31b7aaaf918e3d4221354019bf4a216768570e98ba4 not found: ID does not exist" containerID="18bbb12ec7ffebd2133be31b7aaaf918e3d4221354019bf4a216768570e98ba4"
Oct 31 04:12:43.472545 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.472489    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18bbb12ec7ffebd2133be31b7aaaf918e3d4221354019bf4a216768570e98ba4"} err="failed to get container status \"18bbb12ec7ffebd2133be31b7aaaf918e3d4221354019bf4a216768570e98ba4\": rpc error: code = NotFound desc = could not find container \"18bbb12ec7ffebd2133be31b7aaaf918e3d4221354019bf4a216768570e98ba4\": container with ID starting with 18bbb12ec7ffebd2133be31b7aaaf918e3d4221354019bf4a216768570e98ba4 not found: ID does not exist"
Oct 31 04:12:43.480156 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.480137    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.480156 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.480157    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.480259 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.480166    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.480259 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.480174    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.480259 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.480182    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.480259 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.480206    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.480259 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.480216    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdn8x\" (UniqueName: \"kubernetes.io/projected/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-kube-api-access-rdn8x\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.480259 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.480224    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a5f198d8-39a2-4c72-87de-516e077d47bd-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.480259 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.480232    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.480259 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.480240    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5699efa-62d9-4cb3-b53b-bd83adff2f1d-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.480259 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.480248    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.480259 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.480256    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a5f198d8-39a2-4c72-87de-516e077d47bd-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.480601 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.480278    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj7gn\" (UniqueName: \"kubernetes.io/projected/a5f198d8-39a2-4c72-87de-516e077d47bd-kube-api-access-tj7gn\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.480601 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.480288    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.480601 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.480296    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cfgh\" (UniqueName: \"kubernetes.io/projected/f97017f6-52c4-4d27-9aa7-5045b9ac85ee-kube-api-access-8cfgh\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:43.759740 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.759692    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"]
Oct 31 04:12:43.765710 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.765685    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/simulator-w6-412ba5-kserve-849678b49d-jf46j"]
Oct 31 04:12:43.877780 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.877747    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"]
Oct 31 04:12:43.881625 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.881605    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/premium-w6-412ba5-kserve-6ffcbf948c-5n46l"]
Oct 31 04:12:43.994830 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.994803    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"]
Oct 31 04:12:43.998470 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:43.998446    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-412ba5/distinct-w6-412ba5-kserve-f6dc8dcf4-rq6k7"]
Oct 31 04:12:45.536404 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:45.536372    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5f198d8-39a2-4c72-87de-516e077d47bd" path="/var/lib/kubelet/pods/a5f198d8-39a2-4c72-87de-516e077d47bd/volumes"
Oct 31 04:12:45.536792 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:45.536766    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" path="/var/lib/kubelet/pods/e5699efa-62d9-4cb3-b53b-bd83adff2f1d/volumes"
Oct 31 04:12:45.537110 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:45.537096    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f97017f6-52c4-4d27-9aa7-5045b9ac85ee" path="/var/lib/kubelet/pods/f97017f6-52c4-4d27-9aa7-5045b9ac85ee/volumes"
Oct 31 04:12:53.249227 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.249187    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"]
Oct 31 04:12:53.249777 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.249469    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx" podUID="b73c02cf-bb22-4571-b20b-5445ac0f9c8b" containerName="maas-api" containerID="cri-o://725919db86e5b5fdf957dbc964a9928bc78c58d6739b03027bdf473e6c376928" gracePeriod=30
Oct 31 04:12:53.491772 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.491594    4926 generic.go:356] "Generic (PLEG): container finished" podID="b73c02cf-bb22-4571-b20b-5445ac0f9c8b" containerID="725919db86e5b5fdf957dbc964a9928bc78c58d6739b03027bdf473e6c376928" exitCode=0
Oct 31 04:12:53.491772 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.491628    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx" event={"ID":"b73c02cf-bb22-4571-b20b-5445ac0f9c8b","Type":"ContainerDied","Data":"725919db86e5b5fdf957dbc964a9928bc78c58d6739b03027bdf473e6c376928"}
Oct 31 04:12:53.511517 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.511462    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 04:12:53.567988 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.567947    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-kube-api-access-fzw4x\" (UniqueName: \"kubernetes.io/projected/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-kube-api-access-fzw4x\") pod \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") "
Oct 31 04:12:53.568163 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.568009    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-metrics-certs\") pod \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") "
Oct 31 04:12:53.568163 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.568060    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-maas-api-tls\") pod \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") "
Oct 31 04:12:53.568234 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.568161    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-openshift-service-ca\") pod \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\" (UID: \"b73c02cf-bb22-4571-b20b-5445ac0f9c8b\") "
Oct 31 04:12:53.568565 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.568532    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-openshift-service-ca" pod "b73c02cf-bb22-4571-b20b-5445ac0f9c8b" (UID: "b73c02cf-bb22-4571-b20b-5445ac0f9c8b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:12:53.570323 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.570300    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-kube-api-access-fzw4x" pod "b73c02cf-bb22-4571-b20b-5445ac0f9c8b" (UID: "b73c02cf-bb22-4571-b20b-5445ac0f9c8b"). InnerVolumeSpecName "kube-api-access-fzw4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:12:53.570323 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.570307    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-metrics-certs" pod "b73c02cf-bb22-4571-b20b-5445ac0f9c8b" (UID: "b73c02cf-bb22-4571-b20b-5445ac0f9c8b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:12:53.570453 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.570328    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-maas-api-tls" pod "b73c02cf-bb22-4571-b20b-5445ac0f9c8b" (UID: "b73c02cf-bb22-4571-b20b-5445ac0f9c8b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:12:53.669068 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.669035    4926 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-metrics-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:53.669068 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.669064    4926 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-maas-api-tls\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:53.669068 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.669075    4926 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-openshift-service-ca\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:53.669307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:53.669083    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzw4x\" (UniqueName: \"kubernetes.io/projected/b73c02cf-bb22-4571-b20b-5445ac0f9c8b-kube-api-access-fzw4x\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:12:54.497389 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:54.497351    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx" event={"ID":"b73c02cf-bb22-4571-b20b-5445ac0f9c8b","Type":"ContainerDied","Data":"e41c520f86e122365a7a2538b34c7cbc861e3557571586e5907778c72367e81b"}
Oct 31 04:12:54.497389 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:54.497387    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"
Oct 31 04:12:54.497891 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:54.497423    4926 scope.go:122] "RemoveContainer" containerID="725919db86e5b5fdf957dbc964a9928bc78c58d6739b03027bdf473e6c376928"
Oct 31 04:12:54.666395 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:54.666360    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"]
Oct 31 04:12:54.717821 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:54.717794    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-412ba5-646697bf98-xflwx"]
Oct 31 04:12:55.536828 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:12:55.536782    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b73c02cf-bb22-4571-b20b-5445ac0f9c8b" path="/var/lib/kubelet/pods/b73c02cf-bb22-4571-b20b-5445ac0f9c8b/volumes"
Oct 31 04:13:05.314482 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:13:05.314437    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:13:23.832251 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:23.832225    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:13:23.837496 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:23.837473    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:13:35.311445 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:13:35.311411    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.834415    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"]
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835236    4926 cpu_manager.go:414] "containerMap: removing container" podUID="170d646b-12d9-47eb-88db-232869e6854e" containerName="istio-proxy"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835256    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="170d646b-12d9-47eb-88db-232869e6854e" containerName="istio-proxy"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835313    4926 cpu_manager.go:414] "containerMap: removing container" podUID="e20732a1-da5b-4e22-b861-6b03dfe62522" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835320    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e20732a1-da5b-4e22-b861-6b03dfe62522" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835352    4926 cpu_manager.go:414] "containerMap: removing container" podUID="a5f198d8-39a2-4c72-87de-516e077d47bd" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835360    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5f198d8-39a2-4c72-87de-516e077d47bd" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835378    4926 cpu_manager.go:414] "containerMap: removing container" podUID="b73c02cf-bb22-4571-b20b-5445ac0f9c8b" containerName="maas-api"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835385    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b73c02cf-bb22-4571-b20b-5445ac0f9c8b" containerName="maas-api"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835397    4926 cpu_manager.go:414] "containerMap: removing container" podUID="f97017f6-52c4-4d27-9aa7-5045b9ac85ee" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835403    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f97017f6-52c4-4d27-9aa7-5045b9ac85ee" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835432    4926 cpu_manager.go:414] "containerMap: removing container" podUID="8e545c1f-0b6f-4e47-a54a-238137e641b7" containerName="test-curl-6968-07b6b7"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835439    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e545c1f-0b6f-4e47-a54a-238137e641b7" containerName="test-curl-6968-07b6b7"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835455    4926 cpu_manager.go:414] "containerMap: removing container" podUID="067339e5-0df5-421d-8b14-471a9b1a9735" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835461    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="067339e5-0df5-421d-8b14-471a9b1a9735" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835484    4926 cpu_manager.go:414] "containerMap: removing container" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerName="istio-proxy"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835490    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerName="istio-proxy"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835508    4926 cpu_manager.go:414] "containerMap: removing container" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835515    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835533    4926 cpu_manager.go:414] "containerMap: removing container" podUID="e430bd98-6cc2-4b3a-a521-71a3e2547fdc" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835540    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e430bd98-6cc2-4b3a-a521-71a3e2547fdc" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835709    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="d41233ae-9c3e-466a-aa39-075d14ba5e0f" containerName="istio-proxy"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835728    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="e5699efa-62d9-4cb3-b53b-bd83adff2f1d" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835738    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="e430bd98-6cc2-4b3a-a521-71a3e2547fdc" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835756    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="170d646b-12d9-47eb-88db-232869e6854e" containerName="istio-proxy"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835776    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="e20732a1-da5b-4e22-b861-6b03dfe62522" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835788    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5f198d8-39a2-4c72-87de-516e077d47bd" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835803    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="b73c02cf-bb22-4571-b20b-5445ac0f9c8b" containerName="maas-api"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835812    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="f97017f6-52c4-4d27-9aa7-5045b9ac85ee" containerName="main"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835829    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e545c1f-0b6f-4e47-a54a-238137e641b7" containerName="test-curl-6968-07b6b7"
Oct 31 04:13:46.840237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.835836    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="067339e5-0df5-421d-8b14-471a9b1a9735" containerName="main"
Oct 31 04:13:46.841626 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.841330    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:46.942402 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.942367    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e465810-07db-4c11-96e4-b7057a038c0d-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:46.942608 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.942420    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:46.942608 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.942490    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:46.942608 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.942559    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mllh9\" (UniqueName: \"kubernetes.io/projected/8e465810-07db-4c11-96e4-b7057a038c0d-kube-api-access-mllh9\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:46.942608 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.942580    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:46.942859 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.942683    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:46.942859 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.942718    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:46.942859 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.942759    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e465810-07db-4c11-96e4-b7057a038c0d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:46.942859 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.942818    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e465810-07db-4c11-96e4-b7057a038c0d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:46.960618 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:46.960579    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"]
Oct 31 04:13:47.044431 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.044397    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e465810-07db-4c11-96e4-b7057a038c0d-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.044595 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.044443    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.044595 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.044557    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.044681 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.044599    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mllh9\" (UniqueName: \"kubernetes.io/projected/8e465810-07db-4c11-96e4-b7057a038c0d-kube-api-access-mllh9\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.044681 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.044633    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.044752 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.044694    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.044752 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.044723    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.044828 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.044767    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e465810-07db-4c11-96e4-b7057a038c0d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.044828 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.044794    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e465810-07db-4c11-96e4-b7057a038c0d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.044828 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.044802    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-workload-certs\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.044960 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.044908    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-istio-data\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.045066 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.045040    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-credential-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.045152 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.045068    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-workload-socket\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.045346 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.045329    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e465810-07db-4c11-96e4-b7057a038c0d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.047293 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.047250    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-istio-envoy\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.047407 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.047353    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e465810-07db-4c11-96e4-b7057a038c0d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.108533 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.108465    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e465810-07db-4c11-96e4-b7057a038c0d-istio-token\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.108803 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.108777    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mllh9\" (UniqueName: \"kubernetes.io/projected/8e465810-07db-4c11-96e4-b7057a038c0d-kube-api-access-mllh9\") pod \"maas-default-gateway-openshift-default-7854949c9-nckk9\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.154174 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.154144    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:47.571454 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:13:47.571420    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e465810_07db_4c11_96e4_b7057a038c0d.slice/crio-0cc29bdc3cc79d3e698cfa755194d3d9894f316c74fd62c15f02242a7c2bf7b8 WatchSource:0}: Error finding container 0cc29bdc3cc79d3e698cfa755194d3d9894f316c74fd62c15f02242a7c2bf7b8: Status 404 returned error can't find the container with id 0cc29bdc3cc79d3e698cfa755194d3d9894f316c74fd62c15f02242a7c2bf7b8
Oct 31 04:13:47.573609 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.573577    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 04:13:47.573695 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.573649    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 04:13:47.573695 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.573677    4926 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 31 04:13:47.573978 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.573953    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"]
Oct 31 04:13:47.726753 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.726716    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" event={"ID":"8e465810-07db-4c11-96e4-b7057a038c0d","Type":"ContainerStarted","Data":"22f06604dce2949f9e803bf032a7a712a8ceefed12e20f6a5fe5ffb4affa8e0b"}
Oct 31 04:13:47.726949 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.726783    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" event={"ID":"8e465810-07db-4c11-96e4-b7057a038c0d","Type":"ContainerStarted","Data":"0cc29bdc3cc79d3e698cfa755194d3d9894f316c74fd62c15f02242a7c2bf7b8"}
Oct 31 04:13:47.851093 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:47.851002    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" podStartSLOduration=1.85098805 podStartE2EDuration="1.85098805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:13:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:13:47.849435741 +0000 UTC m=+2724.434976005" watchObservedRunningTime="2026-10-31 04:13:47.85098805 +0000 UTC m=+2724.436528290"
Oct 31 04:13:48.154495 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:48.154405    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:48.156034 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:48.156013    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-nckk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 31 04:13:48.156089 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:48.156060    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" podUID="8e465810-07db-4c11-96e4-b7057a038c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 31 04:13:49.155083 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:49.155046    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-nckk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 31 04:13:49.155552 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:49.155098    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" podUID="8e465810-07db-4c11-96e4-b7057a038c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 31 04:13:50.154981 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:50.154945    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-nckk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 31 04:13:50.155172 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:50.154998    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" podUID="8e465810-07db-4c11-96e4-b7057a038c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 31 04:13:51.154966 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:51.154930    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-nckk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 31 04:13:51.155153 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:51.154997    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" podUID="8e465810-07db-4c11-96e4-b7057a038c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 31 04:13:52.154991 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:52.154955    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-nckk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 31 04:13:52.155419 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:52.155019    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" podUID="8e465810-07db-4c11-96e4-b7057a038c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 31 04:13:53.154750 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:53.154712    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-nckk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 31 04:13:53.155093 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:53.154812    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" podUID="8e465810-07db-4c11-96e4-b7057a038c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 31 04:13:54.154865 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:54.154816    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-nckk9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Oct 31 04:13:54.155109 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:54.154885    4926 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" podUID="8e465810-07db-4c11-96e4-b7057a038c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Oct 31 04:13:55.803117 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:55.803081    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:55.803646 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:55.803470    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:13:55.804316 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:13:55.804290    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:14:05.312847 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:14:05.312760    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:14:35.311419 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:14:35.311383    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:15:05.312847 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:15:05.312808    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:15:35.312021 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:15:35.311977    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:16:05.311609 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:16:05.311571    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:16:11.847293 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:16:11.846378    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-nckk9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:16:11.847293 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:16:11.846448    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" podUID="8e465810-07db-4c11-96e4-b7057a038c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:16:35.312524 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:16:35.312484    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:17:05.312030 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:17:05.311941    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:17:35.317987 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:17:35.317946    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:18:05.313312 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:18:05.313255    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:18:23.861527 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:18:23.861499    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:18:23.873616 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:18:23.873584    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:18:35.312676 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:18:35.312634    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:19:05.312233 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:19:05.312191    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:19:35.313010 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:19:35.312969    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:20:05.315511 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:20:05.315399    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:20:35.311711 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:20:35.311677    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:21:05.312366 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:21:05.312240    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:21:35.310769 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:21:35.310723    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:21:57.088147 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.088116    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"]
Oct 31 04:21:57.091484 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.091468    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.146373 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.146340    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"]
Oct 31 04:21:57.164865 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.164836    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-tls-certs\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.164865 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.164867    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-tmp-dir\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.165044 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.164898    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c82br\" (UniqueName: \"kubernetes.io/projected/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-kube-api-access-c82br\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.165044 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.165016    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-home\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.165195 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.165046    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-model-cache\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.165195 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.165067    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-dshm\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.196194 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.196164    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-f2eca2\"/\"simulator-single-main-f2eca2-kserve-self-signed-certs\""
Oct 31 04:21:57.197103 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.197087    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-f2eca2\"/\"kube-root-ca.crt\""
Oct 31 04:21:57.197163 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.197093    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-f2eca2\"/\"default-dockercfg-fgxns\""
Oct 31 04:21:57.197163 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.197093    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-f2eca2\"/\"openshift-service-ca.crt\""
Oct 31 04:21:57.265583 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.265549    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c82br\" (UniqueName: \"kubernetes.io/projected/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-kube-api-access-c82br\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.265758 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.265603    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-home\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.265758 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.265622    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-model-cache\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.265758 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.265638    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-dshm\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.265758 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.265682    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-tls-certs\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.265758 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.265696    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-tmp-dir\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.266071 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.265986    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-home\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.266117 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.266101    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-tmp-dir\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.266151 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.266123    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-model-cache\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.268196 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.268166    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-dshm\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.268413 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.268398    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-tls-certs\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.328280 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.328239    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c82br\" (UniqueName: \"kubernetes.io/projected/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-kube-api-access-c82br\") pod \"simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") " pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.403343 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.403243    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:21:57.771771 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:21:57.771743    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7767b2c_79eb_4248_92e2_3b1dcd2fff46.slice/crio-3ddf83e926efe156ba69702b17b68f297b61b4614efed20b4de27ce15cf74ffb WatchSource:0}: Error finding container 3ddf83e926efe156ba69702b17b68f297b61b4614efed20b4de27ce15cf74ffb: Status 404 returned error can't find the container with id 3ddf83e926efe156ba69702b17b68f297b61b4614efed20b4de27ce15cf74ffb
Oct 31 04:21:57.773733 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.773707    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"]
Oct 31 04:21:57.964068 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.964032    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z" event={"ID":"e7767b2c-79eb-4248-92e2-3b1dcd2fff46","Type":"ContainerStarted","Data":"405acae03cdb105417a2c2554a12b1f55df67b455c65af7b1ab7c3c1f0d67dc3"}
Oct 31 04:21:57.964233 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:57.964078    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z" event={"ID":"e7767b2c-79eb-4248-92e2-3b1dcd2fff46","Type":"ContainerStarted","Data":"3ddf83e926efe156ba69702b17b68f297b61b4614efed20b4de27ce15cf74ffb"}
Oct 31 04:21:58.085094 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:58.084978    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z" podStartSLOduration=1.084917918 podStartE2EDuration="1.084917918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:21:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:21:58.083424668 +0000 UTC m=+3214.668964930" watchObservedRunningTime="2026-10-31 04:21:58.084917918 +0000 UTC m=+3214.670458180"
Oct 31 04:21:58.403924 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:21:58.403841    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:22:05.312900 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:22:05.312857    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:22:07.403547 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:22:07.403500    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:22:07.416212 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:22:07.416182    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:22:08.021249 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:22:08.021217    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:22:35.311906 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:22:35.311869    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:23:00.101868 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:00.101828    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"]
Oct 31 04:23:05.311057 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:23:05.310979    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:23:15.121495 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.121429    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z" podUID="e7767b2c-79eb-4248-92e2-3b1dcd2fff46" containerName="main" containerID="cri-o://405acae03cdb105417a2c2554a12b1f55df67b455c65af7b1ab7c3c1f0d67dc3" gracePeriod=45
Oct 31 04:23:15.303482 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.294584    4926 generic.go:356] "Generic (PLEG): container finished" podID="e7767b2c-79eb-4248-92e2-3b1dcd2fff46" containerID="405acae03cdb105417a2c2554a12b1f55df67b455c65af7b1ab7c3c1f0d67dc3" exitCode=0
Oct 31 04:23:15.303482 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.294659    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z" event={"ID":"e7767b2c-79eb-4248-92e2-3b1dcd2fff46","Type":"ContainerDied","Data":"405acae03cdb105417a2c2554a12b1f55df67b455c65af7b1ab7c3c1f0d67dc3"}
Oct 31 04:23:15.382849 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.382791    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:23:15.467991 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.467962    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-kube-api-access-c82br\" (UniqueName: \"kubernetes.io/projected/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-kube-api-access-c82br\") pod \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") "
Oct 31 04:23:15.468128 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.468014    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-dshm\") pod \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") "
Oct 31 04:23:15.468128 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.468078    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-home\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-home\") pod \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") "
Oct 31 04:23:15.468128 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.468108    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-tmp-dir\") pod \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") "
Oct 31 04:23:15.468253 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.468155    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-model-cache\") pod \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") "
Oct 31 04:23:15.468253 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.468184    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-tls-certs\" (UniqueName: \"kubernetes.io/secret/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-tls-certs\") pod \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\" (UID: \"e7767b2c-79eb-4248-92e2-3b1dcd2fff46\") "
Oct 31 04:23:15.468408 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.468385    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-home" pod "e7767b2c-79eb-4248-92e2-3b1dcd2fff46" (UID: "e7767b2c-79eb-4248-92e2-3b1dcd2fff46"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:23:15.468467 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.468425    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-tmp-dir" pod "e7767b2c-79eb-4248-92e2-3b1dcd2fff46" (UID: "e7767b2c-79eb-4248-92e2-3b1dcd2fff46"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:23:15.468521 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.468480    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-model-cache" pod "e7767b2c-79eb-4248-92e2-3b1dcd2fff46" (UID: "e7767b2c-79eb-4248-92e2-3b1dcd2fff46"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:23:15.468874 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.468843    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:15.469015 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.468970    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:15.469015 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.468987    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:15.470483 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.470460    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-dshm" pod "e7767b2c-79eb-4248-92e2-3b1dcd2fff46" (UID: "e7767b2c-79eb-4248-92e2-3b1dcd2fff46"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:23:15.470589 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.470494    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-kube-api-access-c82br" pod "e7767b2c-79eb-4248-92e2-3b1dcd2fff46" (UID: "e7767b2c-79eb-4248-92e2-3b1dcd2fff46"). InnerVolumeSpecName "kube-api-access-c82br". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:23:15.470589 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.470508    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-tls-certs" pod "e7767b2c-79eb-4248-92e2-3b1dcd2fff46" (UID: "e7767b2c-79eb-4248-92e2-3b1dcd2fff46"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:23:15.569889 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.569859    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c82br\" (UniqueName: \"kubernetes.io/projected/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-kube-api-access-c82br\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:15.569889 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.569888    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:15.570067 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:15.569902    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7767b2c-79eb-4248-92e2-3b1dcd2fff46-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:23:16.301059 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:16.301027    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z" event={"ID":"e7767b2c-79eb-4248-92e2-3b1dcd2fff46","Type":"ContainerDied","Data":"3ddf83e926efe156ba69702b17b68f297b61b4614efed20b4de27ce15cf74ffb"}
Oct 31 04:23:16.301541 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:16.301077    4926 scope.go:122] "RemoveContainer" containerID="405acae03cdb105417a2c2554a12b1f55df67b455c65af7b1ab7c3c1f0d67dc3"
Oct 31 04:23:16.301541 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:16.301038    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"
Oct 31 04:23:16.470594 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:16.470552    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"]
Oct 31 04:23:16.521853 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:16.521813    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-f2eca2/simulator-single-main-f2eca2-kserve-54fc5cf7fc-k8j2z"]
Oct 31 04:23:17.536510 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:17.536477    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7767b2c-79eb-4248-92e2-3b1dcd2fff46" path="/var/lib/kubelet/pods/e7767b2c-79eb-4248-92e2-3b1dcd2fff46/volumes"
Oct 31 04:23:23.888113 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:23.888083    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:23:23.899753 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:23:23.899686    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:23:35.312167 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:23:35.312129    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:24:05.311416 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:24:05.311380    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:24:35.312035 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:24:35.311997    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:25:05.312400 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:25:05.312364    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:25:19.933439 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:25:19.933385    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mbxvj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:25:19.934212 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:25:19.934162    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" podUID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:25:20.223030 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:25:20.222996    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-kbw7q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:25:20.223229 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:25:20.223086    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q" podUID="a9e112a1-fb10-4386-9700-939a8c0892da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:25:35.312284 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:25:35.312230    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:25:41.858291 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:25:41.857410    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-nckk9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:25:41.858291 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:25:41.857528    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" podUID="8e465810-07db-4c11-96e4-b7057a038c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:26:05.311940 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:26:05.311902    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:26:35.312070 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:26:35.312024    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:26:45.614798 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.614763    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"]
Oct 31 04:26:45.615310 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.615099    4926 cpu_manager.go:414] "containerMap: removing container" podUID="e7767b2c-79eb-4248-92e2-3b1dcd2fff46" containerName="main"
Oct 31 04:26:45.615310 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.615109    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7767b2c-79eb-4248-92e2-3b1dcd2fff46" containerName="main"
Oct 31 04:26:45.615310 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.615187    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7767b2c-79eb-4248-92e2-3b1dcd2fff46" containerName="main"
Oct 31 04:26:45.618210 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.618191    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.673698 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.673657    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"]
Oct 31 04:26:45.723425 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.723386    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-b097c5\"/\"simulator-single-main-b097c5-kserve-self-signed-certs\""
Oct 31 04:26:45.723630 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.723469    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-b097c5\"/\"default-dockercfg-r6t9q\""
Oct 31 04:26:45.723727 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.723711    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-b097c5\"/\"kube-root-ca.crt\""
Oct 31 04:26:45.723787 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.723769    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-b097c5\"/\"openshift-service-ca.crt\""
Oct 31 04:26:45.753417 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.753379    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-model-cache\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.753600 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.753505    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-dshm\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.753600 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.753538    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a6a7d48-6cc3-434c-9362-f334e8027b64-tls-certs\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.753692 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.753632    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-tmp-dir\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.753738 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.753708    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7lsc\" (UniqueName: \"kubernetes.io/projected/1a6a7d48-6cc3-434c-9362-f334e8027b64-kube-api-access-h7lsc\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.753738 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.753727    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-home\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.854747 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.854708    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-dshm\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.854747 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.854746    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a6a7d48-6cc3-434c-9362-f334e8027b64-tls-certs\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.855034 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.854772    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-tmp-dir\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.855034 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.854825    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7lsc\" (UniqueName: \"kubernetes.io/projected/1a6a7d48-6cc3-434c-9362-f334e8027b64-kube-api-access-h7lsc\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.855034 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.854843    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-home\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.855034 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.854862    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-model-cache\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.855355 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.855308    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-tmp-dir\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.855422 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.855356    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-model-cache\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.855562 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.855534    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-home\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.858149 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.858120    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a6a7d48-6cc3-434c-9362-f334e8027b64-tls-certs\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.858400 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.858379    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-dshm\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.918066 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.917991    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7lsc\" (UniqueName: \"kubernetes.io/projected/1a6a7d48-6cc3-434c-9362-f334e8027b64-kube-api-access-h7lsc\") pod \"simulator-single-main-b097c5-kserve-cfb98b684-x46c2\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") " pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:45.929873 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:45.929850    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:46.299126 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:26:46.299089    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a6a7d48_6cc3_434c_9362_f334e8027b64.slice/crio-3c8af1feaaacc1ff5a70d20998113e2f2dc3b1cff24994129325ad5730e49eac WatchSource:0}: Error finding container 3c8af1feaaacc1ff5a70d20998113e2f2dc3b1cff24994129325ad5730e49eac: Status 404 returned error can't find the container with id 3c8af1feaaacc1ff5a70d20998113e2f2dc3b1cff24994129325ad5730e49eac
Oct 31 04:26:46.301363 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:46.301341    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"]
Oct 31 04:26:47.182114 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:47.182078    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2" event={"ID":"1a6a7d48-6cc3-434c-9362-f334e8027b64","Type":"ContainerStarted","Data":"f416df7fb934d613a6b7d95ed0678970b33e532308015bab4fefb77b8ad5b349"}
Oct 31 04:26:47.182523 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:47.182122    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2" event={"ID":"1a6a7d48-6cc3-434c-9362-f334e8027b64","Type":"ContainerStarted","Data":"3c8af1feaaacc1ff5a70d20998113e2f2dc3b1cff24994129325ad5730e49eac"}
Oct 31 04:26:47.345319 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:47.345242    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2" podStartSLOduration=2.345227782 podStartE2EDuration="2.345227782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:26:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:26:47.343821289 +0000 UTC m=+3503.929361551" watchObservedRunningTime="2026-10-31 04:26:47.345227782 +0000 UTC m=+3503.930768044"
Oct 31 04:26:47.930383 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:47.930347    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:55.930699 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:55.930660    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:55.943483 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:55.943452    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:26:56.232716 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:26:56.232691    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:27:05.311661 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:27:05.311622    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:27:35.311741 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:27:35.311704    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:28:05.311993 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:28:05.311946    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:28:07.546319 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:07.546288    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"]
Oct 31 04:28:22.566361 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:22.566321    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2" podUID="1a6a7d48-6cc3-434c-9362-f334e8027b64" containerName="main" containerID="cri-o://f416df7fb934d613a6b7d95ed0678970b33e532308015bab4fefb77b8ad5b349" gracePeriod=45
Oct 31 04:28:22.813686 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:22.813663    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:28:22.942770 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:22.942689    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-model-cache\") pod \"1a6a7d48-6cc3-434c-9362-f334e8027b64\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") "
Oct 31 04:28:22.942922 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:22.942770    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-home\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-home\") pod \"1a6a7d48-6cc3-434c-9362-f334e8027b64\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") "
Oct 31 04:28:22.942922 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:22.942879    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a6a7d48-6cc3-434c-9362-f334e8027b64-kube-api-access-h7lsc\" (UniqueName: \"kubernetes.io/projected/1a6a7d48-6cc3-434c-9362-f334e8027b64-kube-api-access-h7lsc\") pod \"1a6a7d48-6cc3-434c-9362-f334e8027b64\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") "
Oct 31 04:28:22.942922 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:22.942916    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-tmp-dir\") pod \"1a6a7d48-6cc3-434c-9362-f334e8027b64\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") "
Oct 31 04:28:22.943091 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:22.942943    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-dshm\") pod \"1a6a7d48-6cc3-434c-9362-f334e8027b64\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") "
Oct 31 04:28:22.943091 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:22.942975    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a6a7d48-6cc3-434c-9362-f334e8027b64-tls-certs\" (UniqueName: \"kubernetes.io/secret/1a6a7d48-6cc3-434c-9362-f334e8027b64-tls-certs\") pod \"1a6a7d48-6cc3-434c-9362-f334e8027b64\" (UID: \"1a6a7d48-6cc3-434c-9362-f334e8027b64\") "
Oct 31 04:28:22.943570 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:22.943543    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-tmp-dir" pod "1a6a7d48-6cc3-434c-9362-f334e8027b64" (UID: "1a6a7d48-6cc3-434c-9362-f334e8027b64"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:28:22.943758 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:22.943571    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-model-cache" pod "1a6a7d48-6cc3-434c-9362-f334e8027b64" (UID: "1a6a7d48-6cc3-434c-9362-f334e8027b64"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:28:22.943850 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:22.943684    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-home" pod "1a6a7d48-6cc3-434c-9362-f334e8027b64" (UID: "1a6a7d48-6cc3-434c-9362-f334e8027b64"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:28:22.947392 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:22.946971    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a6a7d48-6cc3-434c-9362-f334e8027b64-kube-api-access-h7lsc" pod "1a6a7d48-6cc3-434c-9362-f334e8027b64" (UID: "1a6a7d48-6cc3-434c-9362-f334e8027b64"). InnerVolumeSpecName "kube-api-access-h7lsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:28:22.947392 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:22.947344    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-dshm" pod "1a6a7d48-6cc3-434c-9362-f334e8027b64" (UID: "1a6a7d48-6cc3-434c-9362-f334e8027b64"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:28:22.947545 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:22.947394    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a6a7d48-6cc3-434c-9362-f334e8027b64-tls-certs" pod "1a6a7d48-6cc3-434c-9362-f334e8027b64" (UID: "1a6a7d48-6cc3-434c-9362-f334e8027b64"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:28:23.044586 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.044553    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:23.044755 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.044603    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7lsc\" (UniqueName: \"kubernetes.io/projected/1a6a7d48-6cc3-434c-9362-f334e8027b64-kube-api-access-h7lsc\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:23.044755 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.044614    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:23.044755 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.044622    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:23.044755 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.044630    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a6a7d48-6cc3-434c-9362-f334e8027b64-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:23.044755 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.044639    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a6a7d48-6cc3-434c-9362-f334e8027b64-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:28:23.607793 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.607755    4926 generic.go:356] "Generic (PLEG): container finished" podID="1a6a7d48-6cc3-434c-9362-f334e8027b64" containerID="f416df7fb934d613a6b7d95ed0678970b33e532308015bab4fefb77b8ad5b349" exitCode=0
Oct 31 04:28:23.608249 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.607827    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"
Oct 31 04:28:23.608249 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.607828    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2" event={"ID":"1a6a7d48-6cc3-434c-9362-f334e8027b64","Type":"ContainerDied","Data":"f416df7fb934d613a6b7d95ed0678970b33e532308015bab4fefb77b8ad5b349"}
Oct 31 04:28:23.608249 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.607957    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2" event={"ID":"1a6a7d48-6cc3-434c-9362-f334e8027b64","Type":"ContainerDied","Data":"3c8af1feaaacc1ff5a70d20998113e2f2dc3b1cff24994129325ad5730e49eac"}
Oct 31 04:28:23.608249 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.607987    4926 scope.go:122] "RemoveContainer" containerID="f416df7fb934d613a6b7d95ed0678970b33e532308015bab4fefb77b8ad5b349"
Oct 31 04:28:23.616611 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.616594    4926 scope.go:122] "RemoveContainer" containerID="f416df7fb934d613a6b7d95ed0678970b33e532308015bab4fefb77b8ad5b349"
Oct 31 04:28:23.616868 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:28:23.616846    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f416df7fb934d613a6b7d95ed0678970b33e532308015bab4fefb77b8ad5b349\": container with ID starting with f416df7fb934d613a6b7d95ed0678970b33e532308015bab4fefb77b8ad5b349 not found: ID does not exist" containerID="f416df7fb934d613a6b7d95ed0678970b33e532308015bab4fefb77b8ad5b349"
Oct 31 04:28:23.616938 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.616879    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f416df7fb934d613a6b7d95ed0678970b33e532308015bab4fefb77b8ad5b349"} err="failed to get container status \"f416df7fb934d613a6b7d95ed0678970b33e532308015bab4fefb77b8ad5b349\": rpc error: code = NotFound desc = could not find container \"f416df7fb934d613a6b7d95ed0678970b33e532308015bab4fefb77b8ad5b349\": container with ID starting with f416df7fb934d613a6b7d95ed0678970b33e532308015bab4fefb77b8ad5b349 not found: ID does not exist"
Oct 31 04:28:23.775210 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.775179    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"]
Oct 31 04:28:23.779168 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.779133    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-b097c5/simulator-single-main-b097c5-kserve-cfb98b684-x46c2"]
Oct 31 04:28:23.915170 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.915099    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:28:23.925043 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:23.925012    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:28:25.536414 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:28:25.536384    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a6a7d48-6cc3-434c-9362-f334e8027b64" path="/var/lib/kubelet/pods/1a6a7d48-6cc3-434c-9362-f334e8027b64/volumes"
Oct 31 04:28:35.313228 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:28:35.313185    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:29:05.312943 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:29:05.312856    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:29:05.763095 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.763055    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"]
Oct 31 04:29:05.763468 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.763452    4926 cpu_manager.go:414] "containerMap: removing container" podUID="1a6a7d48-6cc3-434c-9362-f334e8027b64" containerName="main"
Oct 31 04:29:05.763523 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.763469    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a6a7d48-6cc3-434c-9362-f334e8027b64" containerName="main"
Oct 31 04:29:05.763562 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.763552    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a6a7d48-6cc3-434c-9362-f334e8027b64" containerName="main"
Oct 31 04:29:05.766529 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.766511    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.820794 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.820757    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"]
Oct 31 04:29:05.840554 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.840524    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-model-cache\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.840715 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.840575    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-dshm\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.840774 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.840726    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdpdn\" (UniqueName: \"kubernetes.io/projected/e56e842e-003e-4c8b-9153-01402398e1df-kube-api-access-kdpdn\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.840774 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.840759    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-home\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.840857 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.840780    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-tmp-dir\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.840857 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.840804    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56e842e-003e-4c8b-9153-01402398e1df-tls-certs\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.870574 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.870547    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-b433eff5-kserve-self-signed-certs\""
Oct 31 04:29:05.941524 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.941483    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-home\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.941524 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.941527    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-tmp-dir\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.941778 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.941559    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56e842e-003e-4c8b-9153-01402398e1df-tls-certs\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.941778 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.941602    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-model-cache\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.941778 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.941642    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-dshm\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.941778 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.941732    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdpdn\" (UniqueName: \"kubernetes.io/projected/e56e842e-003e-4c8b-9153-01402398e1df-kube-api-access-kdpdn\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.941989 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.941955    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-home\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.942045 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.942001    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-tmp-dir\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.942045 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.942033    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-model-cache\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.944056 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.944029    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-dshm\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:05.944218 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:05.944203    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56e842e-003e-4c8b-9153-01402398e1df-tls-certs\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:06.004895 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:06.004864    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdpdn\" (UniqueName: \"kubernetes.io/projected/e56e842e-003e-4c8b-9153-01402398e1df-kube-api-access-kdpdn\") pod \"e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") " pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:06.078110 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:06.078039    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:06.449524 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:29:06.449482    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode56e842e_003e_4c8b_9153_01402398e1df.slice/crio-2289fd32f848fa7f726913086f009324d8449356a568917488b5a68fa5b5a58e WatchSource:0}: Error finding container 2289fd32f848fa7f726913086f009324d8449356a568917488b5a68fa5b5a58e: Status 404 returned error can't find the container with id 2289fd32f848fa7f726913086f009324d8449356a568917488b5a68fa5b5a58e
Oct 31 04:29:06.451947 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:06.451919    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"]
Oct 31 04:29:06.785536 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:06.785500    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j" event={"ID":"e56e842e-003e-4c8b-9153-01402398e1df","Type":"ContainerStarted","Data":"c809228a2c07252a4dfd327addceef0c9915ae9676f919a4cfa782c167151fa9"}
Oct 31 04:29:06.785715 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:06.785544    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j" event={"ID":"e56e842e-003e-4c8b-9153-01402398e1df","Type":"ContainerStarted","Data":"2289fd32f848fa7f726913086f009324d8449356a568917488b5a68fa5b5a58e"}
Oct 31 04:29:06.910652 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:06.910596    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j" podStartSLOduration=1.910574896 podStartE2EDuration="1.910574896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:29:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:29:06.910376612 +0000 UTC m=+3643.495916873" watchObservedRunningTime="2026-10-31 04:29:06.910574896 +0000 UTC m=+3643.496115197"
Oct 31 04:29:07.078654 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:07.078554    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:16.078605 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:16.078549    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:16.091840 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:16.091816    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:16.863226 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:16.863196    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:29:30.004469 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.004431    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"]
Oct 31 04:29:30.007881 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.007860    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.063237 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.063207    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"]
Oct 31 04:29:30.112966 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.112927    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-b433eff5-kserve-self-signed-certs\""
Oct 31 04:29:30.182898 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.182866    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-dshm\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.183108 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.182915    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-home\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.183108 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.182996    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-tmp-dir\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.183108 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.183052    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e45c192d-3abd-44f0-9c49-c97f6f04e86e-tls-certs\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.183256 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.183121    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-model-cache\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.183256 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.183150    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfx6s\" (UniqueName: \"kubernetes.io/projected/e45c192d-3abd-44f0-9c49-c97f6f04e86e-kube-api-access-cfx6s\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.283853 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.283755    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfx6s\" (UniqueName: \"kubernetes.io/projected/e45c192d-3abd-44f0-9c49-c97f6f04e86e-kube-api-access-cfx6s\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.283853 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.283847    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-dshm\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.284086 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.283903    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-home\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.284086 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.284043    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-tmp-dir\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.284187 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.284106    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e45c192d-3abd-44f0-9c49-c97f6f04e86e-tls-certs\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.284187 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.284136    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-model-cache\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.284456 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.284425    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-home\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.284456 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.284436    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-tmp-dir\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.284615 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.284504    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-model-cache\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.286392 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.286374    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-dshm\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.286668 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.286652    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e45c192d-3abd-44f0-9c49-c97f6f04e86e-tls-certs\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.346526 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.346490    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfx6s\" (UniqueName: \"kubernetes.io/projected/e45c192d-3abd-44f0-9c49-c97f6f04e86e-kube-api-access-cfx6s\") pod \"e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") " pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.618940 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.618855    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:30.991883 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:29:30.991848    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode45c192d_3abd_44f0_9c49_c97f6f04e86e.slice/crio-1dccdd94b9bdfdb7c0a57121d737ae5307387dc52cdc8431cf393cdc5ed44928 WatchSource:0}: Error finding container 1dccdd94b9bdfdb7c0a57121d737ae5307387dc52cdc8431cf393cdc5ed44928: Status 404 returned error can't find the container with id 1dccdd94b9bdfdb7c0a57121d737ae5307387dc52cdc8431cf393cdc5ed44928
Oct 31 04:29:30.993949 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:30.993927    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"]
Oct 31 04:29:31.918880 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:31.918835    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m" event={"ID":"e45c192d-3abd-44f0-9c49-c97f6f04e86e","Type":"ContainerStarted","Data":"ea6c1295b1d4a23f14ea129651be8692e542907c6f4dc00ebd6af0230d7aa417"}
Oct 31 04:29:31.918880 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:31.918881    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m" event={"ID":"e45c192d-3abd-44f0-9c49-c97f6f04e86e","Type":"ContainerStarted","Data":"1dccdd94b9bdfdb7c0a57121d737ae5307387dc52cdc8431cf393cdc5ed44928"}
Oct 31 04:29:32.041169 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:32.041111    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m" podStartSLOduration=3.041097289 podStartE2EDuration="3.041097289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:29:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:29:32.04040795 +0000 UTC m=+3668.625948212" watchObservedRunningTime="2026-10-31 04:29:32.041097289 +0000 UTC m=+3668.626637552"
Oct 31 04:29:32.619127 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:32.619092    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:35.312804 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:29:35.312760    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:29:40.619202 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:40.619161    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:40.631950 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:40.631925    4926 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:29:41.002170 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:29:41.002138    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:30:05.312085 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:30:05.312051    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:30:11.849293 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:11.848390    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-nckk9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:30:11.849293 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:11.848484    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" podUID="8e465810-07db-4c11-96e4-b7057a038c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:30:31.008169 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:31.008133    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"]
Oct 31 04:30:35.315102 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:30:35.315052    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:30:39.939916 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:39.939883    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"]
Oct 31 04:30:46.028860 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.028822    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j" podUID="e56e842e-003e-4c8b-9153-01402398e1df" containerName="main" containerID="cri-o://c809228a2c07252a4dfd327addceef0c9915ae9676f919a4cfa782c167151fa9" gracePeriod=45
Oct 31 04:30:46.287577 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.287516    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:30:46.392421 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.392393    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-dshm\") pod \"e56e842e-003e-4c8b-9153-01402398e1df\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") "
Oct 31 04:30:46.392607 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.392431    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e56e842e-003e-4c8b-9153-01402398e1df-kube-api-access-kdpdn\" (UniqueName: \"kubernetes.io/projected/e56e842e-003e-4c8b-9153-01402398e1df-kube-api-access-kdpdn\") pod \"e56e842e-003e-4c8b-9153-01402398e1df\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") "
Oct 31 04:30:46.392607 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.392490    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e56e842e-003e-4c8b-9153-01402398e1df-tls-certs\" (UniqueName: \"kubernetes.io/secret/e56e842e-003e-4c8b-9153-01402398e1df-tls-certs\") pod \"e56e842e-003e-4c8b-9153-01402398e1df\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") "
Oct 31 04:30:46.392607 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.392517    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-tmp-dir\") pod \"e56e842e-003e-4c8b-9153-01402398e1df\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") "
Oct 31 04:30:46.392607 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.392542    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-model-cache\") pod \"e56e842e-003e-4c8b-9153-01402398e1df\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") "
Oct 31 04:30:46.392607 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.392604    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-home\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-home\") pod \"e56e842e-003e-4c8b-9153-01402398e1df\" (UID: \"e56e842e-003e-4c8b-9153-01402398e1df\") "
Oct 31 04:30:46.392882 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.392812    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-tmp-dir" pod "e56e842e-003e-4c8b-9153-01402398e1df" (UID: "e56e842e-003e-4c8b-9153-01402398e1df"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:30:46.392882 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.392871    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-model-cache" pod "e56e842e-003e-4c8b-9153-01402398e1df" (UID: "e56e842e-003e-4c8b-9153-01402398e1df"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:30:46.392989 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.392892    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-home" pod "e56e842e-003e-4c8b-9153-01402398e1df" (UID: "e56e842e-003e-4c8b-9153-01402398e1df"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:30:46.392989 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.392980    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:46.393123 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.393000    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:46.393123 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.393016    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:46.394926 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.394905    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e56e842e-003e-4c8b-9153-01402398e1df-tls-certs" pod "e56e842e-003e-4c8b-9153-01402398e1df" (UID: "e56e842e-003e-4c8b-9153-01402398e1df"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:30:46.395202 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.395179    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-dshm" pod "e56e842e-003e-4c8b-9153-01402398e1df" (UID: "e56e842e-003e-4c8b-9153-01402398e1df"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:30:46.395202 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.395188    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e56e842e-003e-4c8b-9153-01402398e1df-kube-api-access-kdpdn" pod "e56e842e-003e-4c8b-9153-01402398e1df" (UID: "e56e842e-003e-4c8b-9153-01402398e1df"). InnerVolumeSpecName "kube-api-access-kdpdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:30:46.399815 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.399790    4926 generic.go:356] "Generic (PLEG): container finished" podID="e56e842e-003e-4c8b-9153-01402398e1df" containerID="c809228a2c07252a4dfd327addceef0c9915ae9676f919a4cfa782c167151fa9" exitCode=0
Oct 31 04:30:46.399921 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.399842    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"
Oct 31 04:30:46.399921 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.399882    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j" event={"ID":"e56e842e-003e-4c8b-9153-01402398e1df","Type":"ContainerDied","Data":"c809228a2c07252a4dfd327addceef0c9915ae9676f919a4cfa782c167151fa9"}
Oct 31 04:30:46.400007 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.399924    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j" event={"ID":"e56e842e-003e-4c8b-9153-01402398e1df","Type":"ContainerDied","Data":"2289fd32f848fa7f726913086f009324d8449356a568917488b5a68fa5b5a58e"}
Oct 31 04:30:46.400007 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.399943    4926 scope.go:122] "RemoveContainer" containerID="c809228a2c07252a4dfd327addceef0c9915ae9676f919a4cfa782c167151fa9"
Oct 31 04:30:46.416579 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.416560    4926 scope.go:122] "RemoveContainer" containerID="c809228a2c07252a4dfd327addceef0c9915ae9676f919a4cfa782c167151fa9"
Oct 31 04:30:46.416916 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:30:46.416888    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c809228a2c07252a4dfd327addceef0c9915ae9676f919a4cfa782c167151fa9\": container with ID starting with c809228a2c07252a4dfd327addceef0c9915ae9676f919a4cfa782c167151fa9 not found: ID does not exist" containerID="c809228a2c07252a4dfd327addceef0c9915ae9676f919a4cfa782c167151fa9"
Oct 31 04:30:46.416996 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.416918    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c809228a2c07252a4dfd327addceef0c9915ae9676f919a4cfa782c167151fa9"} err="failed to get container status \"c809228a2c07252a4dfd327addceef0c9915ae9676f919a4cfa782c167151fa9\": rpc error: code = NotFound desc = could not find container \"c809228a2c07252a4dfd327addceef0c9915ae9676f919a4cfa782c167151fa9\": container with ID starting with c809228a2c07252a4dfd327addceef0c9915ae9676f919a4cfa782c167151fa9 not found: ID does not exist"
Oct 31 04:30:46.494131 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.494102    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e56e842e-003e-4c8b-9153-01402398e1df-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:46.494131 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.494128    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdpdn\" (UniqueName: \"kubernetes.io/projected/e56e842e-003e-4c8b-9153-01402398e1df-kube-api-access-kdpdn\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:46.494349 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.494138    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56e842e-003e-4c8b-9153-01402398e1df-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:46.577620 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.577543    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"]
Oct 31 04:30:46.583099 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:46.583073    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-b433eff5-kserve-85447c9545-x7p9j"]
Oct 31 04:30:47.537243 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:47.537215    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e56e842e-003e-4c8b-9153-01402398e1df" path="/var/lib/kubelet/pods/e56e842e-003e-4c8b-9153-01402398e1df/volumes"
Oct 31 04:30:54.959545 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:54.959500    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m" podUID="e45c192d-3abd-44f0-9c49-c97f6f04e86e" containerName="main" containerID="cri-o://ea6c1295b1d4a23f14ea129651be8692e542907c6f4dc00ebd6af0230d7aa417" gracePeriod=45
Oct 31 04:30:54.992036 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:54.992005    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m" podUID="e45c192d-3abd-44f0-9c49-c97f6f04e86e" containerName="main" probeResult="failure" output="Get \"https://10.132.0.93:8000/ready\": dial tcp 10.132.0.93:8000: connect: connection refused"
Oct 31 04:30:55.204468 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.204443    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:30:55.271903 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.271821    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-dshm\") pod \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") "
Oct 31 04:30:55.271903 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.271859    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-home\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-home\") pod \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") "
Oct 31 04:30:55.272129 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.271907    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-model-cache\") pod \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") "
Oct 31 04:30:55.272129 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.271952    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e45c192d-3abd-44f0-9c49-c97f6f04e86e-kube-api-access-cfx6s\" (UniqueName: \"kubernetes.io/projected/e45c192d-3abd-44f0-9c49-c97f6f04e86e-kube-api-access-cfx6s\") pod \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") "
Oct 31 04:30:55.272129 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.271986    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e45c192d-3abd-44f0-9c49-c97f6f04e86e-tls-certs\" (UniqueName: \"kubernetes.io/secret/e45c192d-3abd-44f0-9c49-c97f6f04e86e-tls-certs\") pod \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") "
Oct 31 04:30:55.272129 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.272059    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-tmp-dir\") pod \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\" (UID: \"e45c192d-3abd-44f0-9c49-c97f6f04e86e\") "
Oct 31 04:30:55.272382 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.272205    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-model-cache" pod "e45c192d-3abd-44f0-9c49-c97f6f04e86e" (UID: "e45c192d-3abd-44f0-9c49-c97f6f04e86e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:30:55.272382 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.272215    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-home" pod "e45c192d-3abd-44f0-9c49-c97f6f04e86e" (UID: "e45c192d-3abd-44f0-9c49-c97f6f04e86e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:30:55.272504 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.272382    4926 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-home\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:55.272504 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.272406    4926 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-model-cache\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:55.272504 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.272412    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-tmp-dir" pod "e45c192d-3abd-44f0-9c49-c97f6f04e86e" (UID: "e45c192d-3abd-44f0-9c49-c97f6f04e86e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:30:55.274227 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.274202    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-dshm" pod "e45c192d-3abd-44f0-9c49-c97f6f04e86e" (UID: "e45c192d-3abd-44f0-9c49-c97f6f04e86e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:30:55.274401 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.274245    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e45c192d-3abd-44f0-9c49-c97f6f04e86e-tls-certs" pod "e45c192d-3abd-44f0-9c49-c97f6f04e86e" (UID: "e45c192d-3abd-44f0-9c49-c97f6f04e86e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:30:55.274401 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.274354    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e45c192d-3abd-44f0-9c49-c97f6f04e86e-kube-api-access-cfx6s" pod "e45c192d-3abd-44f0-9c49-c97f6f04e86e" (UID: "e45c192d-3abd-44f0-9c49-c97f6f04e86e"). InnerVolumeSpecName "kube-api-access-cfx6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:30:55.373719 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.373686    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfx6s\" (UniqueName: \"kubernetes.io/projected/e45c192d-3abd-44f0-9c49-c97f6f04e86e-kube-api-access-cfx6s\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:55.373719 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.373712    4926 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e45c192d-3abd-44f0-9c49-c97f6f04e86e-tls-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:55.373719 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.373723    4926 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-tmp-dir\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:55.373948 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.373731    4926 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e45c192d-3abd-44f0-9c49-c97f6f04e86e-dshm\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:30:55.440110 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.440075    4926 generic.go:356] "Generic (PLEG): container finished" podID="e45c192d-3abd-44f0-9c49-c97f6f04e86e" containerID="ea6c1295b1d4a23f14ea129651be8692e542907c6f4dc00ebd6af0230d7aa417" exitCode=0
Oct 31 04:30:55.440303 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.440123    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"
Oct 31 04:30:55.440303 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.440137    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m" event={"ID":"e45c192d-3abd-44f0-9c49-c97f6f04e86e","Type":"ContainerDied","Data":"ea6c1295b1d4a23f14ea129651be8692e542907c6f4dc00ebd6af0230d7aa417"}
Oct 31 04:30:55.440303 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.440170    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m" event={"ID":"e45c192d-3abd-44f0-9c49-c97f6f04e86e","Type":"ContainerDied","Data":"1dccdd94b9bdfdb7c0a57121d737ae5307387dc52cdc8431cf393cdc5ed44928"}
Oct 31 04:30:55.440303 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.440189    4926 scope.go:122] "RemoveContainer" containerID="ea6c1295b1d4a23f14ea129651be8692e542907c6f4dc00ebd6af0230d7aa417"
Oct 31 04:30:55.449570 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.449553    4926 scope.go:122] "RemoveContainer" containerID="ea6c1295b1d4a23f14ea129651be8692e542907c6f4dc00ebd6af0230d7aa417"
Oct 31 04:30:55.449819 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:30:55.449801    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea6c1295b1d4a23f14ea129651be8692e542907c6f4dc00ebd6af0230d7aa417\": container with ID starting with ea6c1295b1d4a23f14ea129651be8692e542907c6f4dc00ebd6af0230d7aa417 not found: ID does not exist" containerID="ea6c1295b1d4a23f14ea129651be8692e542907c6f4dc00ebd6af0230d7aa417"
Oct 31 04:30:55.449863 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.449827    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea6c1295b1d4a23f14ea129651be8692e542907c6f4dc00ebd6af0230d7aa417"} err="failed to get container status \"ea6c1295b1d4a23f14ea129651be8692e542907c6f4dc00ebd6af0230d7aa417\": rpc error: code = NotFound desc = could not find container \"ea6c1295b1d4a23f14ea129651be8692e542907c6f4dc00ebd6af0230d7aa417\": container with ID starting with ea6c1295b1d4a23f14ea129651be8692e542907c6f4dc00ebd6af0230d7aa417 not found: ID does not exist"
Oct 31 04:30:55.609548 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.609462    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"]
Oct 31 04:30:55.613177 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:55.613153    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-b433eff5-kserve-558f4f9b57-nds2m"]
Oct 31 04:30:57.536841 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:30:57.536802    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e45c192d-3abd-44f0-9c49-c97f6f04e86e" path="/var/lib/kubelet/pods/e45c192d-3abd-44f0-9c49-c97f6f04e86e/volumes"
Oct 31 04:31:04.930451 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:04.930409    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mbxvj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 31 04:31:04.931122 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:04.931069    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" podUID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 31 04:31:05.310927 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:31:05.310892    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:31:35.311017 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:31:35.310975    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:31:36.325593 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.325559    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"]
Oct 31 04:31:36.326007 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.325927    4926 cpu_manager.go:414] "containerMap: removing container" podUID="e45c192d-3abd-44f0-9c49-c97f6f04e86e" containerName="main"
Oct 31 04:31:36.326007 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.325939    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e45c192d-3abd-44f0-9c49-c97f6f04e86e" containerName="main"
Oct 31 04:31:36.326007 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.325991    4926 cpu_manager.go:414] "containerMap: removing container" podUID="e56e842e-003e-4c8b-9153-01402398e1df" containerName="main"
Oct 31 04:31:36.326007 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.325997    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e56e842e-003e-4c8b-9153-01402398e1df" containerName="main"
Oct 31 04:31:36.326166 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.326054    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="e56e842e-003e-4c8b-9153-01402398e1df" containerName="main"
Oct 31 04:31:36.326166 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.326069    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="e45c192d-3abd-44f0-9c49-c97f6f04e86e" containerName="main"
Oct 31 04:31:36.333520 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.333504    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:36.384572 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.384544    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"]
Oct 31 04:31:36.460490 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.460465    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b6ec566-2924-4b2c-9379-1f9b35f4c10c-openshift-service-ca\") pod \"maas-api-c8b55458c-gcgmk\" (UID: \"7b6ec566-2924-4b2c-9379-1f9b35f4c10c\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:36.460656 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.460498    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl5zv\" (UniqueName: \"kubernetes.io/projected/7b6ec566-2924-4b2c-9379-1f9b35f4c10c-kube-api-access-dl5zv\") pod \"maas-api-c8b55458c-gcgmk\" (UID: \"7b6ec566-2924-4b2c-9379-1f9b35f4c10c\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:36.460656 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.460646    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b6ec566-2924-4b2c-9379-1f9b35f4c10c-metrics-certs\") pod \"maas-api-c8b55458c-gcgmk\" (UID: \"7b6ec566-2924-4b2c-9379-1f9b35f4c10c\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:36.460739 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.460693    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b6ec566-2924-4b2c-9379-1f9b35f4c10c-maas-api-tls\") pod \"maas-api-c8b55458c-gcgmk\" (UID: \"7b6ec566-2924-4b2c-9379-1f9b35f4c10c\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:36.561253 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.561223    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl5zv\" (UniqueName: \"kubernetes.io/projected/7b6ec566-2924-4b2c-9379-1f9b35f4c10c-kube-api-access-dl5zv\") pod \"maas-api-c8b55458c-gcgmk\" (UID: \"7b6ec566-2924-4b2c-9379-1f9b35f4c10c\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:36.561479 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.561323    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b6ec566-2924-4b2c-9379-1f9b35f4c10c-metrics-certs\") pod \"maas-api-c8b55458c-gcgmk\" (UID: \"7b6ec566-2924-4b2c-9379-1f9b35f4c10c\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:36.561479 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.561341    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b6ec566-2924-4b2c-9379-1f9b35f4c10c-maas-api-tls\") pod \"maas-api-c8b55458c-gcgmk\" (UID: \"7b6ec566-2924-4b2c-9379-1f9b35f4c10c\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:36.561479 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.561366    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b6ec566-2924-4b2c-9379-1f9b35f4c10c-openshift-service-ca\") pod \"maas-api-c8b55458c-gcgmk\" (UID: \"7b6ec566-2924-4b2c-9379-1f9b35f4c10c\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:36.561945 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.561913    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b6ec566-2924-4b2c-9379-1f9b35f4c10c-openshift-service-ca\") pod \"maas-api-c8b55458c-gcgmk\" (UID: \"7b6ec566-2924-4b2c-9379-1f9b35f4c10c\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:36.563936 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.563912    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b6ec566-2924-4b2c-9379-1f9b35f4c10c-metrics-certs\") pod \"maas-api-c8b55458c-gcgmk\" (UID: \"7b6ec566-2924-4b2c-9379-1f9b35f4c10c\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:36.564098 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.564076    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b6ec566-2924-4b2c-9379-1f9b35f4c10c-maas-api-tls\") pod \"maas-api-c8b55458c-gcgmk\" (UID: \"7b6ec566-2924-4b2c-9379-1f9b35f4c10c\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:36.624716 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.624646    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl5zv\" (UniqueName: \"kubernetes.io/projected/7b6ec566-2924-4b2c-9379-1f9b35f4c10c-kube-api-access-dl5zv\") pod \"maas-api-c8b55458c-gcgmk\" (UID: \"7b6ec566-2924-4b2c-9379-1f9b35f4c10c\") " pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:36.646135 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:36.646111    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:37.014932 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:31:37.014901    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b6ec566_2924_4b2c_9379_1f9b35f4c10c.slice/crio-ad32e1573a7ef5dc2b0b298ec229506d4f500064ec04502afa05e3c4408fc4c7 WatchSource:0}: Error finding container ad32e1573a7ef5dc2b0b298ec229506d4f500064ec04502afa05e3c4408fc4c7: Status 404 returned error can't find the container with id ad32e1573a7ef5dc2b0b298ec229506d4f500064ec04502afa05e3c4408fc4c7
Oct 31 04:31:37.016371 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:37.016350    4926 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 31 04:31:37.017003 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:37.016980    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"]
Oct 31 04:31:37.634436 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:37.634402    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk" event={"ID":"7b6ec566-2924-4b2c-9379-1f9b35f4c10c","Type":"ContainerStarted","Data":"ad32e1573a7ef5dc2b0b298ec229506d4f500064ec04502afa05e3c4408fc4c7"}
Oct 31 04:31:38.640727 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:38.640689    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk" event={"ID":"7b6ec566-2924-4b2c-9379-1f9b35f4c10c","Type":"ContainerStarted","Data":"6d4a72a561565c7b728ed175decf30a3c6b1d0436fa2dec170404479cf985e4a"}
Oct 31 04:31:38.641222 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:38.640809    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:38.764629 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:38.764578    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk" podStartSLOduration=1.91463327 podStartE2EDuration="2.764563669s" totalImagesPullingTime="849.930399ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-31 04:31:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:31:38.762963742 +0000 UTC m=+3795.348504004" watchObservedRunningTime="2026-10-31 04:31:38.764563669 +0000 UTC m=+3795.350103981"
Oct 31 04:31:44.649213 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:44.649186    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-c8b55458c-gcgmk"
Oct 31 04:31:44.823673 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:44.823638    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c9679795-4c74x"]
Oct 31 04:31:44.823923 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:44.823892    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x" podUID="0e5af999-10c2-4fe8-ae65-800b96a87896" containerName="maas-api" containerID="cri-o://198cdb9e51bf0e45a9dfb3f981adc778383fe3c62ce407e4c38825196121add1" gracePeriod=30
Oct 31 04:31:46.076381 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.076358    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 04:31:46.249610 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.249574    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e5af999-10c2-4fe8-ae65-800b96a87896-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e5af999-10c2-4fe8-ae65-800b96a87896-maas-api-tls\") pod \"0e5af999-10c2-4fe8-ae65-800b96a87896\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") "
Oct 31 04:31:46.249807 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.249630    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e5af999-10c2-4fe8-ae65-800b96a87896-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e5af999-10c2-4fe8-ae65-800b96a87896-metrics-certs\") pod \"0e5af999-10c2-4fe8-ae65-800b96a87896\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") "
Oct 31 04:31:46.249807 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.249663    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e5af999-10c2-4fe8-ae65-800b96a87896-kube-api-access-867vq\" (UniqueName: \"kubernetes.io/projected/0e5af999-10c2-4fe8-ae65-800b96a87896-kube-api-access-867vq\") pod \"0e5af999-10c2-4fe8-ae65-800b96a87896\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") "
Oct 31 04:31:46.249807 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.249703    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e5af999-10c2-4fe8-ae65-800b96a87896-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e5af999-10c2-4fe8-ae65-800b96a87896-openshift-service-ca\") pod \"0e5af999-10c2-4fe8-ae65-800b96a87896\" (UID: \"0e5af999-10c2-4fe8-ae65-800b96a87896\") "
Oct 31 04:31:46.250128 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.250095    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e5af999-10c2-4fe8-ae65-800b96a87896-openshift-service-ca" pod "0e5af999-10c2-4fe8-ae65-800b96a87896" (UID: "0e5af999-10c2-4fe8-ae65-800b96a87896"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:31:46.251890 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.251860    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e5af999-10c2-4fe8-ae65-800b96a87896-maas-api-tls" pod "0e5af999-10c2-4fe8-ae65-800b96a87896" (UID: "0e5af999-10c2-4fe8-ae65-800b96a87896"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:31:46.251998 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.251903    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e5af999-10c2-4fe8-ae65-800b96a87896-metrics-certs" pod "0e5af999-10c2-4fe8-ae65-800b96a87896" (UID: "0e5af999-10c2-4fe8-ae65-800b96a87896"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 31 04:31:46.251998 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.251936    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e5af999-10c2-4fe8-ae65-800b96a87896-kube-api-access-867vq" pod "0e5af999-10c2-4fe8-ae65-800b96a87896" (UID: "0e5af999-10c2-4fe8-ae65-800b96a87896"). InnerVolumeSpecName "kube-api-access-867vq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:31:46.351351 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.351314    4926 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e5af999-10c2-4fe8-ae65-800b96a87896-openshift-service-ca\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:31:46.351351 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.351344    4926 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e5af999-10c2-4fe8-ae65-800b96a87896-maas-api-tls\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:31:46.351351 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.351354    4926 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e5af999-10c2-4fe8-ae65-800b96a87896-metrics-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:31:46.351593 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.351362    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-867vq\" (UniqueName: \"kubernetes.io/projected/0e5af999-10c2-4fe8-ae65-800b96a87896-kube-api-access-867vq\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:31:46.675339 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.675242    4926 generic.go:356] "Generic (PLEG): container finished" podID="0e5af999-10c2-4fe8-ae65-800b96a87896" containerID="198cdb9e51bf0e45a9dfb3f981adc778383fe3c62ce407e4c38825196121add1" exitCode=0
Oct 31 04:31:46.675339 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.675316    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x" event={"ID":"0e5af999-10c2-4fe8-ae65-800b96a87896","Type":"ContainerDied","Data":"198cdb9e51bf0e45a9dfb3f981adc778383fe3c62ce407e4c38825196121add1"}
Oct 31 04:31:46.675563 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.675339    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x"
Oct 31 04:31:46.675563 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.675388    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68c9679795-4c74x" event={"ID":"0e5af999-10c2-4fe8-ae65-800b96a87896","Type":"ContainerDied","Data":"764652539bfa6052b3f93ebebe0bf58a20f06ac198d2698ede220ccc203df77b"}
Oct 31 04:31:46.675563 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.675419    4926 scope.go:122] "RemoveContainer" containerID="198cdb9e51bf0e45a9dfb3f981adc778383fe3c62ce407e4c38825196121add1"
Oct 31 04:31:46.684890 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.684871    4926 scope.go:122] "RemoveContainer" containerID="198cdb9e51bf0e45a9dfb3f981adc778383fe3c62ce407e4c38825196121add1"
Oct 31 04:31:46.685152 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:31:46.685135    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"198cdb9e51bf0e45a9dfb3f981adc778383fe3c62ce407e4c38825196121add1\": container with ID starting with 198cdb9e51bf0e45a9dfb3f981adc778383fe3c62ce407e4c38825196121add1 not found: ID does not exist" containerID="198cdb9e51bf0e45a9dfb3f981adc778383fe3c62ce407e4c38825196121add1"
Oct 31 04:31:46.685212 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.685162    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"198cdb9e51bf0e45a9dfb3f981adc778383fe3c62ce407e4c38825196121add1"} err="failed to get container status \"198cdb9e51bf0e45a9dfb3f981adc778383fe3c62ce407e4c38825196121add1\": rpc error: code = NotFound desc = could not find container \"198cdb9e51bf0e45a9dfb3f981adc778383fe3c62ce407e4c38825196121add1\": container with ID starting with 198cdb9e51bf0e45a9dfb3f981adc778383fe3c62ce407e4c38825196121add1 not found: ID does not exist"
Oct 31 04:31:46.846763 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.846731    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c9679795-4c74x"]
Oct 31 04:31:46.850903 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:46.850883    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-68c9679795-4c74x"]
Oct 31 04:31:47.536665 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:31:47.536629    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e5af999-10c2-4fe8-ae65-800b96a87896" path="/var/lib/kubelet/pods/0e5af999-10c2-4fe8-ae65-800b96a87896/volumes"
Oct 31 04:32:05.311063 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:32:05.310986    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:32:35.311711 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:32:35.311664    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:33:05.311755 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:33:05.311710    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:33:23.941944 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:33:23.941874    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:33:23.950940 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:33:23.950918    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:33:35.312056 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:33:35.312016    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:34:05.311758 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:34:05.311720    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:34:35.312288 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:34:35.312230    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:35:05.312578 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:35:05.312495    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:35:35.313104 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:35:35.313065    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:36:05.312500 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:36:05.312457    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:36:35.312673 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:36:35.312635    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:36:57.450205 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:36:57.450171    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-jkdzt_7386e379-d04b-44b9-8107-23139ac0f6c6/manager/0.log"
Oct 31 04:37:00.038826 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:37:00.038791    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-c8b55458c-gcgmk_7b6ec566-2924-4b2c-9379-1f9b35f4c10c/maas-api/0.log"
Oct 31 04:37:05.312431 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:37:05.312386    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:37:15.905980 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:37:15.905943    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5897747699-kzfcc_038905c7-ab91-4a97-b514-7cf197c09a31/kube-auth-proxy/0.log"
Oct 31 04:37:17.166935 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:37:17.166902    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7854949c9-kbw7q_a9e112a1-fb10-4386-9700-939a8c0892da/istio-proxy/0.log"
Oct 31 04:37:17.496060 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:37:17.496026    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7854949c9-mbxvj_a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515/istio-proxy/0.log"
Oct 31 04:37:17.774654 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:37:17.774564    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7854949c9-nckk9_8e465810-07db-4c11-96e4-b7057a038c0d/istio-proxy/0.log"
Oct 31 04:37:29.555724 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:37:29.555686    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77685b76bf-cmqhp_26a79eda-be35-4aa8-b10b-cb2b1133a086/router/0.log"
Oct 31 04:37:35.312579 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:37:35.312543    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:37:42.291409 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:37:42.291378    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4_6fc591ce-8543-4590-9dd5-9002226418d2/main/0.log"
Oct 31 04:38:05.311973 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:38:05.311889    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:38:14.279668 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:38:14.279637    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-jkdzt_7386e379-d04b-44b9-8107-23139ac0f6c6/manager/0.log"
Oct 31 04:38:16.582670 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:38:16.582635    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-c8b55458c-gcgmk_7b6ec566-2924-4b2c-9379-1f9b35f4c10c/maas-api/0.log"
Oct 31 04:38:23.970673 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:38:23.970645    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:38:23.978072 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:38:23.978050    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-wpxfm_bb5340b5-c677-40d1-a448-498561c62a17/console-operator/2.log"
Oct 31 04:38:35.312375 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:38:35.312333    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:38:42.346373 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:38:42.346337    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5897747699-kzfcc_038905c7-ab91-4a97-b514-7cf197c09a31/kube-auth-proxy/0.log"
Oct 31 04:38:43.632383 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:38:43.632354    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7854949c9-kbw7q_a9e112a1-fb10-4386-9700-939a8c0892da/istio-proxy/0.log"
Oct 31 04:38:43.981843 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:38:43.981812    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7854949c9-mbxvj_a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515/istio-proxy/0.log"
Oct 31 04:38:44.291008 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:38:44.290940    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7854949c9-nckk9_8e465810-07db-4c11-96e4-b7057a038c0d/istio-proxy/0.log"
Oct 31 04:38:56.142037 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:38:56.142001    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-77685b76bf-cmqhp_26a79eda-be35-4aa8-b10b-cb2b1133a086/router/0.log"
Oct 31 04:39:05.312714 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:39:05.312671    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:39:08.718049 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:39:08.718008    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-lm7x4_6fc591ce-8543-4590-9dd5-9002226418d2/main/0.log"
Oct 31 04:39:35.311725 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:39:35.311683    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:40:05.311673 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:40:05.311633    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:40:35.312758 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:40:35.312721    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:41:05.311948 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:41:05.311870    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:41:35.312307 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:41:35.312252    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:41:58.842936 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:41:58.842894    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"]
Oct 31 04:41:58.845526 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:41:58.843146    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" podUID="8e465810-07db-4c11-96e4-b7057a038c0d" containerName="istio-proxy" containerID="cri-o://22f06604dce2949f9e803bf032a7a712a8ceefed12e20f6a5fe5ffb4affa8e0b" gracePeriod=30
Oct 31 04:42:04.092907 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.092883    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:42:04.158330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158242    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e465810-07db-4c11-96e4-b7057a038c0d-kube-api-access-mllh9\" (UniqueName: \"kubernetes.io/projected/8e465810-07db-4c11-96e4-b7057a038c0d-kube-api-access-mllh9\") pod \"8e465810-07db-4c11-96e4-b7057a038c0d\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") "
Oct 31 04:42:04.158330 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158308    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-credential-socket\") pod \"8e465810-07db-4c11-96e4-b7057a038c0d\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") "
Oct 31 04:42:04.158516 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158372    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e465810-07db-4c11-96e4-b7057a038c0d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e465810-07db-4c11-96e4-b7057a038c0d-istiod-ca-cert\") pod \"8e465810-07db-4c11-96e4-b7057a038c0d\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") "
Oct 31 04:42:04.158516 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158411    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-workload-certs\") pod \"8e465810-07db-4c11-96e4-b7057a038c0d\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") "
Oct 31 04:42:04.158516 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158446    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-workload-socket\") pod \"8e465810-07db-4c11-96e4-b7057a038c0d\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") "
Oct 31 04:42:04.158516 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158463    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-istio-data\") pod \"8e465810-07db-4c11-96e4-b7057a038c0d\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") "
Oct 31 04:42:04.158742 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158534    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-istio-envoy\") pod \"8e465810-07db-4c11-96e4-b7057a038c0d\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") "
Oct 31 04:42:04.158742 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158588    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e465810-07db-4c11-96e4-b7057a038c0d-istio-token\" (UniqueName: \"kubernetes.io/projected/8e465810-07db-4c11-96e4-b7057a038c0d-istio-token\") pod \"8e465810-07db-4c11-96e4-b7057a038c0d\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") "
Oct 31 04:42:04.158742 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158598    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-credential-socket" pod "8e465810-07db-4c11-96e4-b7057a038c0d" (UID: "8e465810-07db-4c11-96e4-b7057a038c0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.158742 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158618    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8e465810-07db-4c11-96e4-b7057a038c0d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e465810-07db-4c11-96e4-b7057a038c0d-istio-podinfo\") pod \"8e465810-07db-4c11-96e4-b7057a038c0d\" (UID: \"8e465810-07db-4c11-96e4-b7057a038c0d\") "
Oct 31 04:42:04.158742 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158707    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-workload-certs" pod "8e465810-07db-4c11-96e4-b7057a038c0d" (UID: "8e465810-07db-4c11-96e4-b7057a038c0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.158742 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158716    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-istio-data" pod "8e465810-07db-4c11-96e4-b7057a038c0d" (UID: "8e465810-07db-4c11-96e4-b7057a038c0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.158994 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158737    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-workload-socket" pod "8e465810-07db-4c11-96e4-b7057a038c0d" (UID: "8e465810-07db-4c11-96e4-b7057a038c0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.158994 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158797    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e465810-07db-4c11-96e4-b7057a038c0d-istiod-ca-cert" pod "8e465810-07db-4c11-96e4-b7057a038c0d" (UID: "8e465810-07db-4c11-96e4-b7057a038c0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:42:04.158994 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158960    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8e465810-07db-4c11-96e4-b7057a038c0d-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.158994 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158979    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.158994 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.158993    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.159288 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.159006    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.159288 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.159018    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.160734 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.160712    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-istio-envoy" pod "8e465810-07db-4c11-96e4-b7057a038c0d" (UID: "8e465810-07db-4c11-96e4-b7057a038c0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:04.160848 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.160745    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e465810-07db-4c11-96e4-b7057a038c0d-kube-api-access-mllh9" pod "8e465810-07db-4c11-96e4-b7057a038c0d" (UID: "8e465810-07db-4c11-96e4-b7057a038c0d"). InnerVolumeSpecName "kube-api-access-mllh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:42:04.160892 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.160869    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e465810-07db-4c11-96e4-b7057a038c0d-istio-token" pod "8e465810-07db-4c11-96e4-b7057a038c0d" (UID: "8e465810-07db-4c11-96e4-b7057a038c0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:42:04.160892 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.160879    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8e465810-07db-4c11-96e4-b7057a038c0d-istio-podinfo" pod "8e465810-07db-4c11-96e4-b7057a038c0d" (UID: "8e465810-07db-4c11-96e4-b7057a038c0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:42:04.259598 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.259562    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mllh9\" (UniqueName: \"kubernetes.io/projected/8e465810-07db-4c11-96e4-b7057a038c0d-kube-api-access-mllh9\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.259598 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.259597    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8e465810-07db-4c11-96e4-b7057a038c0d-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.259893 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.259612    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8e465810-07db-4c11-96e4-b7057a038c0d-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.259893 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.259625    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8e465810-07db-4c11-96e4-b7057a038c0d-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:04.327665 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.327624    4926 generic.go:356] "Generic (PLEG): container finished" podID="8e465810-07db-4c11-96e4-b7057a038c0d" containerID="22f06604dce2949f9e803bf032a7a712a8ceefed12e20f6a5fe5ffb4affa8e0b" exitCode=0
Oct 31 04:42:04.327860 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.327699    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"
Oct 31 04:42:04.327860 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.327702    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" event={"ID":"8e465810-07db-4c11-96e4-b7057a038c0d","Type":"ContainerDied","Data":"22f06604dce2949f9e803bf032a7a712a8ceefed12e20f6a5fe5ffb4affa8e0b"}
Oct 31 04:42:04.327860 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.327829    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9" event={"ID":"8e465810-07db-4c11-96e4-b7057a038c0d","Type":"ContainerDied","Data":"0cc29bdc3cc79d3e698cfa755194d3d9894f316c74fd62c15f02242a7c2bf7b8"}
Oct 31 04:42:04.327860 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.327858    4926 scope.go:122] "RemoveContainer" containerID="22f06604dce2949f9e803bf032a7a712a8ceefed12e20f6a5fe5ffb4affa8e0b"
Oct 31 04:42:04.337402 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.337377    4926 scope.go:122] "RemoveContainer" containerID="22f06604dce2949f9e803bf032a7a712a8ceefed12e20f6a5fe5ffb4affa8e0b"
Oct 31 04:42:04.337735 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:42:04.337710    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22f06604dce2949f9e803bf032a7a712a8ceefed12e20f6a5fe5ffb4affa8e0b\": container with ID starting with 22f06604dce2949f9e803bf032a7a712a8ceefed12e20f6a5fe5ffb4affa8e0b not found: ID does not exist" containerID="22f06604dce2949f9e803bf032a7a712a8ceefed12e20f6a5fe5ffb4affa8e0b"
Oct 31 04:42:04.337839 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.337741    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22f06604dce2949f9e803bf032a7a712a8ceefed12e20f6a5fe5ffb4affa8e0b"} err="failed to get container status \"22f06604dce2949f9e803bf032a7a712a8ceefed12e20f6a5fe5ffb4affa8e0b\": rpc error: code = NotFound desc = could not find container \"22f06604dce2949f9e803bf032a7a712a8ceefed12e20f6a5fe5ffb4affa8e0b\": container with ID starting with 22f06604dce2949f9e803bf032a7a712a8ceefed12e20f6a5fe5ffb4affa8e0b not found: ID does not exist"
Oct 31 04:42:04.347154 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:42:04.347130    4926 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e465810_07db_4c11_96e4_b7057a038c0d.slice/crio-0cc29bdc3cc79d3e698cfa755194d3d9894f316c74fd62c15f02242a7c2bf7b8\": RecentStats: unable to find data in memory cache]"
Oct 31 04:42:04.496247 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.496211    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"]
Oct 31 04:42:04.504739 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:04.504703    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-nckk9"]
Oct 31 04:42:05.311863 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:42:05.311825    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:42:05.537788 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:05.537756    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e465810-07db-4c11-96e4-b7057a038c0d" path="/var/lib/kubelet/pods/8e465810-07db-4c11-96e4-b7057a038c0d/volumes"
Oct 31 04:42:13.807837 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:13.807802    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"]
Oct 31 04:42:13.808375 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:13.808026    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q" podUID="a9e112a1-fb10-4386-9700-939a8c0892da" containerName="istio-proxy" containerID="cri-o://41d5c1e4a7ce9706e185da695eaef7964f1a48d4f877fb846d84fa6b05f65f50" gracePeriod=30
Oct 31 04:42:13.860764 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:13.860730    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"]
Oct 31 04:42:13.861462 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:13.861429    4926 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" podUID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" containerName="istio-proxy" containerID="cri-o://39ef6d55bd02c6c373302661d5be7069e79f72c5782008a859484044c80a61be" gracePeriod=30
Oct 31 04:42:18.888010 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:18.887967    4926 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7854949c9-mbxvj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Oct 31 04:42:18.888397 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:18.888027    4926 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" podUID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Oct 31 04:42:19.126787 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.126758    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 04:42:19.143572 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.143551    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 04:42:19.291554 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.291511    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-token\" (UniqueName: \"kubernetes.io/projected/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-token\") pod \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") "
Oct 31 04:42:19.291554 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.291557    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a9e112a1-fb10-4386-9700-939a8c0892da-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e112a1-fb10-4386-9700-939a8c0892da-istio-podinfo\") pod \"a9e112a1-fb10-4386-9700-939a8c0892da\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") "
Oct 31 04:42:19.291815 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.291583    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-workload-certs\") pod \"a9e112a1-fb10-4386-9700-939a8c0892da\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") "
Oct 31 04:42:19.291815 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.291599    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9e112a1-fb10-4386-9700-939a8c0892da-istio-token\" (UniqueName: \"kubernetes.io/projected/a9e112a1-fb10-4386-9700-939a8c0892da-istio-token\") pod \"a9e112a1-fb10-4386-9700-939a8c0892da\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") "
Oct 31 04:42:19.291815 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.291733    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-envoy\") pod \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") "
Oct 31 04:42:19.291815 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.291790    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9e112a1-fb10-4386-9700-939a8c0892da-kube-api-access-vs2cn\" (UniqueName: \"kubernetes.io/projected/a9e112a1-fb10-4386-9700-939a8c0892da-kube-api-access-vs2cn\") pod \"a9e112a1-fb10-4386-9700-939a8c0892da\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") "
Oct 31 04:42:19.292038 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.291830    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-credential-socket\") pod \"a9e112a1-fb10-4386-9700-939a8c0892da\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") "
Oct 31 04:42:19.292038 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.291874    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-workload-socket\") pod \"a9e112a1-fb10-4386-9700-939a8c0892da\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") "
Oct 31 04:42:19.292038 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.291882    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-workload-certs" pod "a9e112a1-fb10-4386-9700-939a8c0892da" (UID: "a9e112a1-fb10-4386-9700-939a8c0892da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:19.292038 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.291907    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9e112a1-fb10-4386-9700-939a8c0892da-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e112a1-fb10-4386-9700-939a8c0892da-istiod-ca-cert\") pod \"a9e112a1-fb10-4386-9700-939a8c0892da\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") "
Oct 31 04:42:19.292038 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.291949    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-workload-certs\") pod \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") "
Oct 31 04:42:19.292038 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.291979    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-istio-data\") pod \"a9e112a1-fb10-4386-9700-939a8c0892da\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") "
Oct 31 04:42:19.292038 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.292018    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-credential-socket\") pod \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") "
Oct 31 04:42:19.292425 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.292049    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-podinfo\") pod \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") "
Oct 31 04:42:19.292425 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.292103    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-data\") pod \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") "
Oct 31 04:42:19.292425 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.292115    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-credential-socket" pod "a9e112a1-fb10-4386-9700-939a8c0892da" (UID: "a9e112a1-fb10-4386-9700-939a8c0892da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:19.292425 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.292139    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istiod-ca-cert\") pod \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") "
Oct 31 04:42:19.292425 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.292170    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-istio-envoy\") pod \"a9e112a1-fb10-4386-9700-939a8c0892da\" (UID: \"a9e112a1-fb10-4386-9700-939a8c0892da\") "
Oct 31 04:42:19.292425 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.292207    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-kube-api-access-r84zg\" (UniqueName: \"kubernetes.io/projected/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-kube-api-access-r84zg\") pod \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") "
Oct 31 04:42:19.292425 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.292235    4926 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-workload-socket\") pod \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\" (UID: \"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515\") "
Oct 31 04:42:19.292425 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.292365    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-istio-data" pod "a9e112a1-fb10-4386-9700-939a8c0892da" (UID: "a9e112a1-fb10-4386-9700-939a8c0892da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:19.292852 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.292585    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-workload-socket" pod "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" (UID: "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:19.292852 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.292784    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-credential-socket" pod "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" (UID: "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:19.294307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.292990    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istiod-ca-cert" pod "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" (UID: "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:42:19.294307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.293333    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9e112a1-fb10-4386-9700-939a8c0892da-istiod-ca-cert" pod "a9e112a1-fb10-4386-9700-939a8c0892da" (UID: "a9e112a1-fb10-4386-9700-939a8c0892da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 31 04:42:19.294307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.293595    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-data" pod "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" (UID: "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:19.294307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.293647    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-workload-certs" pod "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" (UID: "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:19.294307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.293877    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.294307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.293898    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e112a1-fb10-4386-9700-939a8c0892da-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.294307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.293916    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.294307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.293931    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.294307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.293947    4926 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-credential-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.294307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.293962    4926 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-data\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.294307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.293977    4926 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istiod-ca-cert\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.294307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.293993    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.294307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.294007    4926 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-workload-certs\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.294307 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.294228    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-workload-socket" pod "a9e112a1-fb10-4386-9700-939a8c0892da" (UID: "a9e112a1-fb10-4386-9700-939a8c0892da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:19.295161 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.294552    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-token" pod "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" (UID: "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:42:19.295161 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.294571    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9e112a1-fb10-4386-9700-939a8c0892da-istio-token" pod "a9e112a1-fb10-4386-9700-939a8c0892da" (UID: "a9e112a1-fb10-4386-9700-939a8c0892da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:42:19.295161 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.294650    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-envoy" pod "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" (UID: "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:19.295161 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.294859    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9e112a1-fb10-4386-9700-939a8c0892da-kube-api-access-vs2cn" pod "a9e112a1-fb10-4386-9700-939a8c0892da" (UID: "a9e112a1-fb10-4386-9700-939a8c0892da"). InnerVolumeSpecName "kube-api-access-vs2cn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:42:19.295161 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.294977    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a9e112a1-fb10-4386-9700-939a8c0892da-istio-podinfo" pod "a9e112a1-fb10-4386-9700-939a8c0892da" (UID: "a9e112a1-fb10-4386-9700-939a8c0892da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:42:19.295578 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.295555    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-istio-envoy" pod "a9e112a1-fb10-4386-9700-939a8c0892da" (UID: "a9e112a1-fb10-4386-9700-939a8c0892da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 31 04:42:19.295763 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.295745    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-podinfo" pod "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" (UID: "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 31 04:42:19.296064 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.296052    4926 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-kube-api-access-r84zg" pod "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" (UID: "a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515"). InnerVolumeSpecName "kube-api-access-r84zg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 31 04:42:19.389712 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.389624    4926 generic.go:356] "Generic (PLEG): container finished" podID="a9e112a1-fb10-4386-9700-939a8c0892da" containerID="41d5c1e4a7ce9706e185da695eaef7964f1a48d4f877fb846d84fa6b05f65f50" exitCode=0
Oct 31 04:42:19.389712 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.389679    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"
Oct 31 04:42:19.389960 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.389716    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q" event={"ID":"a9e112a1-fb10-4386-9700-939a8c0892da","Type":"ContainerDied","Data":"41d5c1e4a7ce9706e185da695eaef7964f1a48d4f877fb846d84fa6b05f65f50"}
Oct 31 04:42:19.389960 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.389810    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q" event={"ID":"a9e112a1-fb10-4386-9700-939a8c0892da","Type":"ContainerDied","Data":"d83b766588c38a9166f418c0da94ebd6f6c61225efbc510f69cb0d33c826cfab"}
Oct 31 04:42:19.389960 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.389842    4926 scope.go:122] "RemoveContainer" containerID="41d5c1e4a7ce9706e185da695eaef7964f1a48d4f877fb846d84fa6b05f65f50"
Oct 31 04:42:19.391114 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.391092    4926 generic.go:356] "Generic (PLEG): container finished" podID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" containerID="39ef6d55bd02c6c373302661d5be7069e79f72c5782008a859484044c80a61be" exitCode=0
Oct 31 04:42:19.391230 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.391160    4926 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"
Oct 31 04:42:19.391230 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.391175    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" event={"ID":"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515","Type":"ContainerDied","Data":"39ef6d55bd02c6c373302661d5be7069e79f72c5782008a859484044c80a61be"}
Oct 31 04:42:19.391230 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.391217    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj" event={"ID":"a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515","Type":"ContainerDied","Data":"a7945332f3cbf2e0f783f17b1a39f3e6fceccc3dae73d3261f2b1045b4a4cdb9"}
Oct 31 04:42:19.394879 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.394855    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.395007 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.394883    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.395007 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.394898    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r84zg\" (UniqueName: \"kubernetes.io/projected/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-kube-api-access-r84zg\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.395007 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.394912    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.395007 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.394926    4926 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9e112a1-fb10-4386-9700-939a8c0892da-istio-podinfo\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.395007 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.394938    4926 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9e112a1-fb10-4386-9700-939a8c0892da-istio-token\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.395007 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.394952    4926 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515-istio-envoy\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.395007 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.394967    4926 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vs2cn\" (UniqueName: \"kubernetes.io/projected/a9e112a1-fb10-4386-9700-939a8c0892da-kube-api-access-vs2cn\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.395007 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.394980    4926 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9e112a1-fb10-4386-9700-939a8c0892da-workload-socket\") on node \"ip-10-0-130-19.us-west-2.compute.internal\" DevicePath \"\""
Oct 31 04:42:19.398575 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.398550    4926 scope.go:122] "RemoveContainer" containerID="41d5c1e4a7ce9706e185da695eaef7964f1a48d4f877fb846d84fa6b05f65f50"
Oct 31 04:42:19.398842 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:42:19.398820    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41d5c1e4a7ce9706e185da695eaef7964f1a48d4f877fb846d84fa6b05f65f50\": container with ID starting with 41d5c1e4a7ce9706e185da695eaef7964f1a48d4f877fb846d84fa6b05f65f50 not found: ID does not exist" containerID="41d5c1e4a7ce9706e185da695eaef7964f1a48d4f877fb846d84fa6b05f65f50"
Oct 31 04:42:19.398932 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.398852    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41d5c1e4a7ce9706e185da695eaef7964f1a48d4f877fb846d84fa6b05f65f50"} err="failed to get container status \"41d5c1e4a7ce9706e185da695eaef7964f1a48d4f877fb846d84fa6b05f65f50\": rpc error: code = NotFound desc = could not find container \"41d5c1e4a7ce9706e185da695eaef7964f1a48d4f877fb846d84fa6b05f65f50\": container with ID starting with 41d5c1e4a7ce9706e185da695eaef7964f1a48d4f877fb846d84fa6b05f65f50 not found: ID does not exist"
Oct 31 04:42:19.398932 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.398875    4926 scope.go:122] "RemoveContainer" containerID="39ef6d55bd02c6c373302661d5be7069e79f72c5782008a859484044c80a61be"
Oct 31 04:42:19.407003 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.406984    4926 scope.go:122] "RemoveContainer" containerID="39ef6d55bd02c6c373302661d5be7069e79f72c5782008a859484044c80a61be"
Oct 31 04:42:19.407238 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:42:19.407220    4926 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39ef6d55bd02c6c373302661d5be7069e79f72c5782008a859484044c80a61be\": container with ID starting with 39ef6d55bd02c6c373302661d5be7069e79f72c5782008a859484044c80a61be not found: ID does not exist" containerID="39ef6d55bd02c6c373302661d5be7069e79f72c5782008a859484044c80a61be"
Oct 31 04:42:19.407350 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.407245    4926 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39ef6d55bd02c6c373302661d5be7069e79f72c5782008a859484044c80a61be"} err="failed to get container status \"39ef6d55bd02c6c373302661d5be7069e79f72c5782008a859484044c80a61be\": rpc error: code = NotFound desc = could not find container \"39ef6d55bd02c6c373302661d5be7069e79f72c5782008a859484044c80a61be\": container with ID starting with 39ef6d55bd02c6c373302661d5be7069e79f72c5782008a859484044c80a61be not found: ID does not exist"
Oct 31 04:42:19.539255 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.539223    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"]
Oct 31 04:42:19.541350 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.541327    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-kbw7q"]
Oct 31 04:42:19.653298 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.653209    4926 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"]
Oct 31 04:42:19.659826 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:19.659803    4926 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7854949c9-mbxvj"]
Oct 31 04:42:21.537282 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:21.537231    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9e112a1-fb10-4386-9700-939a8c0892da" path="/var/lib/kubelet/pods/a9e112a1-fb10-4386-9700-939a8c0892da/volumes"
Oct 31 04:42:21.537671 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:21.537661    4926 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" path="/var/lib/kubelet/pods/a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515/volumes"
Oct 31 04:42:35.311820 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:42:35.311778    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:42:45.097529 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:45.097496    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-phb2z_609f4320-8b1b-459e-9e9b-9aeeed31dbe2/global-pull-secret-syncer/0.log"
Oct 31 04:42:46.281118 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:46.281092    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-8pxph_bf18ddb4-723c-4954-8327-37f38b41eff1/connection-checker/0.log"
Oct 31 04:42:46.706346 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:46.706319    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-htdq6_cad00ddf-f3ee-48a4-8dbc-52786cb8bc6a/connection-checker/0.log"
Oct 31 04:42:47.107609 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:47.107531    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-kfthp_02dec91e-45a1-4609-9b09-72ba5259bd0e/connection-checker/0.log"
Oct 31 04:42:48.108674 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:48.108636    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qwh92_a0883de4-ab87-4a25-bd8b-ecd4914d4638/konnectivity-agent/0.log"
Oct 31 04:42:49.303822 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:49.303785    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-19.us-west-2.compute.internal_8cce19b2c56183ccd97ae9b3e0b6ff57/haproxy/0.log"
Oct 31 04:42:54.660095 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.660062    4926 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"]
Oct 31 04:42:54.660499 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.660399    4926 cpu_manager.go:414] "containerMap: removing container" podUID="a9e112a1-fb10-4386-9700-939a8c0892da" containerName="istio-proxy"
Oct 31 04:42:54.660499 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.660412    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9e112a1-fb10-4386-9700-939a8c0892da" containerName="istio-proxy"
Oct 31 04:42:54.660499 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.660428    4926 cpu_manager.go:414] "containerMap: removing container" podUID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" containerName="istio-proxy"
Oct 31 04:42:54.660499 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.660433    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" containerName="istio-proxy"
Oct 31 04:42:54.660499 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.660443    4926 cpu_manager.go:414] "containerMap: removing container" podUID="8e465810-07db-4c11-96e4-b7057a038c0d" containerName="istio-proxy"
Oct 31 04:42:54.660499 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.660449    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e465810-07db-4c11-96e4-b7057a038c0d" containerName="istio-proxy"
Oct 31 04:42:54.660499 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.660455    4926 cpu_manager.go:414] "containerMap: removing container" podUID="0e5af999-10c2-4fe8-ae65-800b96a87896" containerName="maas-api"
Oct 31 04:42:54.660499 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.660460    4926 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e5af999-10c2-4fe8-ae65-800b96a87896" containerName="maas-api"
Oct 31 04:42:54.660754 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.660511    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9e112a1-fb10-4386-9700-939a8c0892da" containerName="istio-proxy"
Oct 31 04:42:54.660754 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.660521    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9e8b83e-ccbd-4f1f-86a3-4f0cf2025515" containerName="istio-proxy"
Oct 31 04:42:54.660754 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.660528    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e465810-07db-4c11-96e4-b7057a038c0d" containerName="istio-proxy"
Oct 31 04:42:54.660754 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.660533    4926 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e5af999-10c2-4fe8-ae65-800b96a87896" containerName="maas-api"
Oct 31 04:42:54.663539 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.663517    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.716955 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.716926    4926 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xddgj\"/\"default-dockercfg-x59w7\""
Oct 31 04:42:54.764076 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.764047    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xddgj\"/\"openshift-service-ca.crt\""
Oct 31 04:42:54.764188 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.764086    4926 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xddgj\"/\"kube-root-ca.crt\""
Oct 31 04:42:54.769813 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.769792    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"]
Oct 31 04:42:54.787983 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.787960    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1-sys\") pod \"perf-node-gather-daemonset-jh7zw\" (UID: \"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.788095 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.787993    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1-lib-modules\") pod \"perf-node-gather-daemonset-jh7zw\" (UID: \"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.788095 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.788015    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1-proc\") pod \"perf-node-gather-daemonset-jh7zw\" (UID: \"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.788190 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.788143    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1-podres\") pod \"perf-node-gather-daemonset-jh7zw\" (UID: \"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.788190 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.788183    4926 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mxkr\" (UniqueName: \"kubernetes.io/projected/d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1-kube-api-access-2mxkr\") pod \"perf-node-gather-daemonset-jh7zw\" (UID: \"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.889309 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.889256    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1-podres\") pod \"perf-node-gather-daemonset-jh7zw\" (UID: \"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.889478 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.889328    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mxkr\" (UniqueName: \"kubernetes.io/projected/d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1-kube-api-access-2mxkr\") pod \"perf-node-gather-daemonset-jh7zw\" (UID: \"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.889478 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.889387    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1-sys\") pod \"perf-node-gather-daemonset-jh7zw\" (UID: \"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.889478 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.889388    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1-podres\") pod \"perf-node-gather-daemonset-jh7zw\" (UID: \"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.889478 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.889443    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1-sys\") pod \"perf-node-gather-daemonset-jh7zw\" (UID: \"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.889614 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.889501    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1-lib-modules\") pod \"perf-node-gather-daemonset-jh7zw\" (UID: \"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.889614 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.889521    4926 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1-proc\") pod \"perf-node-gather-daemonset-jh7zw\" (UID: \"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.889614 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.889596    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1-proc\") pod \"perf-node-gather-daemonset-jh7zw\" (UID: \"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.889614 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.889602    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1-lib-modules\") pod \"perf-node-gather-daemonset-jh7zw\" (UID: \"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.951991 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.951961    4926 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mxkr\" (UniqueName: \"kubernetes.io/projected/d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1-kube-api-access-2mxkr\") pod \"perf-node-gather-daemonset-jh7zw\" (UID: \"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1\") " pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:54.975029 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:54.975005    4926 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:55.345121 ip-10-0-130-19 kubenswrapper[4926]: W1031 04:42:55.345080    4926 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd2cb2b56_99a2_475a_b8c6_dd8bca84f0c1.slice/crio-169fbb90750e502ec79f1fb9792b2d6ce6d3e6f02c8bf2a3d2e7dc7bedab9774 WatchSource:0}: Error finding container 169fbb90750e502ec79f1fb9792b2d6ce6d3e6f02c8bf2a3d2e7dc7bedab9774: Status 404 returned error can't find the container with id 169fbb90750e502ec79f1fb9792b2d6ce6d3e6f02c8bf2a3d2e7dc7bedab9774
Oct 31 04:42:55.347874 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:55.347850    4926 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"]
Oct 31 04:42:55.544780 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:55.544749    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw" event={"ID":"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1","Type":"ContainerStarted","Data":"873e58554fc4c458c4978011842cf56f03be0ef883331de2ca6ca577b09da2f1"}
Oct 31 04:42:55.544948 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:55.544819    4926 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw" event={"ID":"d2cb2b56-99a2-475a-b8c6-dd8bca84f0c1","Type":"ContainerStarted","Data":"169fbb90750e502ec79f1fb9792b2d6ce6d3e6f02c8bf2a3d2e7dc7bedab9774"}
Oct 31 04:42:55.544948 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:55.544886    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:42:55.664002 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:42:55.663903    4926 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw" podStartSLOduration=1.66389128 podStartE2EDuration="1.66389128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-31 04:42:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-31 04:42:55.662885444 +0000 UTC m=+4472.248425707" watchObservedRunningTime="2026-10-31 04:42:55.66389128 +0000 UTC m=+4472.249431541"
Oct 31 04:43:00.856447 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:43:00.856413    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-2dwx2_aef16625-d836-42cb-bf49-176f80e73474/cluster-monitoring-operator/0.log"
Oct 31 04:43:01.559547 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:43:01.559521    4926 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xddgj/perf-node-gather-daemonset-jh7zw"
Oct 31 04:43:03.218730 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:43:03.218702    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-glhsr_b801defd-0f62-417c-b440-f5c8e8635701/node-exporter/0.log"
Oct 31 04:43:03.605744 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:43:03.605659    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-glhsr_b801defd-0f62-417c-b440-f5c8e8635701/kube-rbac-proxy/0.log"
Oct 31 04:43:03.993564 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:43:03.993537    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-glhsr_b801defd-0f62-417c-b440-f5c8e8635701/init-textfile/0.log"
Oct 31 04:43:05.311872 ip-10-0-130-19 kubenswrapper[4926]: E1031 04:43:05.311838    4926 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-tw5k4" containerName="extractor"
Oct 31 04:43:17.804328 ip-10-0-130-19 kubenswrapper[4926]: I1031 04:43:17.804293    4926 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-k2fcf_f4b87fa6-eceb-4783-b2c5-4685e08ba9ef/networking-console-plugin/0.log"